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About This Guide 

Introduction 

Using data to make decisions about students is challenging. For many years, educators have been asked 
to use their professional judgment, without readily available data, to determine student needs. The 
amount of data available to inform these decisions has increased exponentially; still, the sheer amount 
and access to these data continue to make using data to identify student needs a challenge. The 
Massachusetts Department of Elementary and Secondary Education (ESE) is focused on supporting 
educators in the use of data; to this end, ESE developed an Early Warning Indicator System (EWIS) that 
rolls many student data variables into a single indicator that provides educators with information about 
which students are at low, moderate, or high risk of missing key academic milestones as they enter 
school each fall. 

Annually, the Massachusetts EWIS provides information about students' risk levels for every student in 
grades 1-12 who attended a public school in Massachusetts in the year prior. EWIS is based on several 
years of research that examined student data and identified the combination of data points that most 
accurately predict the likelihood that a student will reach or fall short of achieving a key educational 
milestone. The risk levels of students are provided to Massachusetts' districts and schools in the late 
summer each year. EWIS data provides educators with critical information about whether students are 
on track for the next important academic milestone. Identifying students early on in the school year 
allows educators to take action to make schoolwide changes to minimize risk, provide appropriate 
interventions and supports to identified students when they first enter school in the fall, and monitor 
student progress throughout the school year to ensure that students stay on track. 

Purpose of the Guide 

The purpose of this guide is to provide information on how to use early warning data, including the 
Massachusetts Early Warning Indicator System (EWIS), to identify, diagnose, support and monitor 
students in grades 1-12. It offers educators an overview of EWIS and how to effectively use these data in 
conjunction with local data by following a six-step implementation cycle. 

Because no two districts, schools, or students are alike, we purposely developed this guide as a flexible 
tool to help educators customize their use of EWIS to fit their individual needs, interests, and resources. 
We hope that this guide will help you with the important work you do every day in your classrooms, 
schools, and districts. 
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This guide is divided into the following four main sections: 


■ Section 1. What are Early Warning Data? This section defines early warning data and provides 
background information on the Massachusetts EWIS, including how it was developed, how it works, 
and instructions for accessing EWIS data. Districts/schools new to EWIS will find this section 
particularly helpful for understanding EWIS, and getting ready to use early warning data for school 
improvement and student support purposes. 

■ Section 2. The Six-Step Early Warning Implementation Process. This section introduces schools and 
districts to a six-step early warning implementation process and is organized into subsections for 
each step in the cycle. Each subsection briefly describes the step, anticipated outcomes, and the 
tasks and processes schools and districts can use to implement the step. "Notes from the Field" 
anecdotes derived from Massachusetts districts and schools that have piloted an early warning 
implementation process, are included to offer readers promising approaches, lessons learned and 
concrete examples for implementation from varying district and school contexts. “Build off Your 
Foundation" includes prompts that encourage schools and districts to build upon the systems and 
practices that are already in place when using early warning data. Each step includes a set of 
guiding questions to help you move through the implementation process and outlines some of the 
ways in which districts can support early warning data use. Finally, each step concludes with some 
tools that may be helpful for implementation of the step. 

■ Section 3. Additional Resources. This section provides information and links to additional 
Massachusetts and other resources that may be beneficial. 

■ Section 4. Appendixes. This section provides additional information to support implementation and 
understanding of early warning data. 
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Section 1. What are Early Warning Data? 


Across the United States, districts and schools are using 
early warning data to predict the likelihood that a student 
will meet or miss key academic milestones, such as 
graduating from high school. Research has found that 
readily available student data sources such as attendance, 
behavioral records, and course failures, can be used as early 
warning indicators to identify students who are at academic 
risk. For example, data on student attendance, when 
combined with a research-based threshold such as missing 
10% of school days, can be used to "flag" students who are 
at risk of not graduating from high school. Early warning 
data also includes locally determined indicators and diagnostic data, such as local benchmark 
assessments and demographic information, that schools and districts use to identify and diagnose the 
needs of students, provide interventions and supports, and track student progress throughout the year. 
Districts and schools also use early warning data to inform schoolwide strategies to address student 
needs, and to examine long-term, school-level patterns to address systemic issues that may impede a 
school's or a student's ability to meet academic milestones. Although a key aspect of using early warning 
data is the identification of students, it is essential that such identification does not result in another 
label for students but rather in actionable information that leads to support for students in need. (For 
more information on early warning data, see http://www.earlvwarningsystems.org/services/resources- 
tools#proj2-tab ). In Massachusetts these data are integral to the implementation of the Massachusetts 
Tiered System of Support (MTSS): http://www.doe.mass.edu/mtss/ 

About the Massachusetts EWIS 

The Massachusetts EWIS is an early warning system that identifies students who are at risk of missing 
key academic milestones in grades 1-12. EWIS student risk levels are computed annually at the end of 
each summer as students begin the school year. The Massachusetts EWIS was developed in direct 
response to educators' requests for information that enables them to identify students who may be at 
risk at earlier grade levels and throughout high school. Working with American Institutes for Research 
(AIR), ESE intentionally designed EWIS to rely solely on data available through existing statewide 
collections of multiple data sources. Using a rigorous statistical method, researchers created a valid 
EWIS model for each grade level, from elementary school through high school. 2 The EWIS development 
process was several years in the making. The EWIS risk models reflect national research and promising 
practices. Extensive testing of possible data sources, and statistical modeling using longitudinal 
Massachusetts data, were used to create three student risk levels that comprise the current EWIS. (For 
more information on the development of the EWIS model and risk levels, see Appendix A or 
http://www.doe.mass.edu/edwin/analytics/ewis.html for the technical documentation.) 


Implementation Tip : 

Learn More About EWIS 

Is your district or school new (or relatively 
new) to the use of early warning data? This 
section of the Guide will be especially 
helpful in preparing you and your team to 
understand EWIS and be ready to use early 
warning data in your school or district. 
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EWIS Academic Milestones 


Massachusetts EWIS student risk levels are predictive of relevant academic milestones. A student's EWIS 
risk level predicts the likelihood that the student will or will not meet a particular academic milestone. 

In EWIS, student risk is organized by four grade level groupings. These age groups are as follows: early 
elementary, late elementary, middle grades, and high school. For each age group, a student's EWIS risk 
level predicts the likelihood that the student will or will not meet an academic milestone identified for 
that particular age group. The academic milestones that have been identified for each age group are 
developmentally appropriate, based on available state data, important to the success of Massachusetts' 
students, and meaningful and actionable for adult educators who work with the students in each grade 
grouping (Exhibit 1). For example, the early elementary age group encompasses grades 1-3 and assesses 
risk based on the academic milestone of achieving a score of Proficient or Advanced on the grade 3 
English Language Arts (ELA) state assessment test. Scoring Proficient or Advanced on the grade 3 ELA 
State Assessment is a proxy for reading by the end of grade 3 and is a developmentally appropriate 
benchmark for children in the early grades. 


Exhibit 1. Massachusetts Age Groups, Grade Levels, and Academic milestones 
or Milestones 


Age Groups 

Grade 

Levels 

Academic milestone 

(Expected Student Outcome for Each Age Group) 

Early elementary 

1-3 

Reading by the end of grade 3: Proficient or higher on the grade 3 ELA 

State Assessment 

Late elementary 

4-6 

Middle school ready: Proficient or higher on the grade 6 ELA and 

mathematics State Assessment 

Middle grades 

7-9 

High school ready: Passing grades in all grade 9 courses 

High school 

10-12 

High school graduation: Completing all local and state graduation 
requirements in four years. 


The academic milestones for early elementary, late elementary, and the middle grades also are positively 
associated with high school graduation (the high school academic milestone). For example, research points to 
the importance of reading by the end of grade 3 as an important milestone for students to be on track for 
ultimately graduating from high school. 3 Students who do not score proficient on the third grade state 
assessment are 3.7 times less likely to graduate from high school. 

EWIS Student Risk Levels 

There are three risk levels in EWIS: low, moderate, and high. These risk levels relate to a student's predicted 
likelihood for reaching his or her age group's defined academic milestone. A risk level quickly provides 
information to educators about the likelihood that an individual student will or will not achieve an academic 
milestone. In other words, the risk level indicates whether a student is currently on track to reach the 
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upcoming academic milestone. For example, a student who is identified as low risk is predicted as likely to 
meet the academic milestone. 


The EWIS risk levels are determined by using data from the previous school year. A risk level is assigned 
to every Massachusetts public school student for whom there are state-level data from the prior year. 
The risk levels are determined on an individual student basis and are not based on a student's relative 
likelihood of reaching an academic milestone when compared with other students. As a result, there are 
no set amounts of students at each risk level. For example, it is possible to have all students in the low- 
risk category. Exhibit 2 overviews the EWIS risk levels. 


Exhibit 2. EWIS Student Risk Levels 


Low Risk 

Likelv to reach the upcoming academic milestone. Approximately 90 percent of 

students who are at low risk will meet this academic milestone within each aae 

group. 

Moderate Risk 

Moderately at risk for not reaching the upcoming academic milestone. 

Approximately 60 percent of students at moderate risk meet this academic 

milestone within each age group. 

High Risk 

At risk for not reaching the upcoming academic milestone. Approximately 

25 percent of students at hiah risk meet this academic milestone within each aae 

group. 


For more information about EWIS, including the data included to determine risk and the background of 
the development of the EWIS, see Appendix A. 


Accessing EWIS Data 

EWIS data for Massachusetts students are available to 
districts and schools through Edwin Analytics 1 . ESE developed 
a range of EWIS reports to facilitate educators' use of this 
information. EWIS data also can be exported to Excel and 
combined with other student data sources. EWIS data are 
made available in late summer of each school year and 
provide information on each student who attended a 
Massachusetts public school in the prior year. Districts and 
schools can access EWIS data through Edwin Analytics, which 
is found by using the ESE website security portal, available through the drop-down menu on the upper 
right side of the ESE website: http://www.doe.mass.edu/ . The data also can be directly accessed at 


Implementation Tip : 

Accessing EWIS Data 

■ Access EWIS reports in Edwin 
Analytics: 

https://gateway.edu.state.ma.us/ 

* Contact your district's directory 
administrator to request access: 

http://www.doe.mass.edu/infoservices 

/data/diradmin/list.aspx 


1 Edwin is a comprehensive system that includes Edwin Analytics, available to all MA districts. 
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https://gateway.edu.state.ma.us/ . Each district's directory administrator provide staff access to Edwin 
Analytics. The list of directory administrators can be found at 

http://www.doe.mass.edu/infoservices/data/diradmin/list.aspx . For more information about using Edwin 
and the EWIS data and reports, see Appendix B or http://www.doe.mass.edu/edwin/analytics/ . 

About Monitoring Indicators 

EWIS student risk levels offer districts and schools information on students based on the prior year's 
data. Monitoring indicators, which use readily available local data, can complement the use of EWIS 
data. After receiving the EWIS risk levels for students at the beginning of the year, monitoring indicators 
are a means by which educators can track student progress and flag students as in need during the 
course of a school year. For example, a grade 2 student who is identified by EWIS as being at high risk of 
not scoring Proficient or higher on the grade 3 ELA State Assessment may be provided with reading 
support at the beginning of the school year. The team may then use attendance, disciplinary data, and 
reading benchmark data as monitoring indicators several times during the year to make sure this 
student stays on track. In addition to using monitoring indicators to observe the progress of individual 
students, districts and schools may use monitoring indicators to identify trends in student needs and make 
recommendations for schoolwide improvements such as changes in schedules, curriculum, or 
instructional approaches. Tool 3 in Step 1 overviews potential monitoring indicators and indicates 
whether they are research based or commonly used but need to be locally validated. Teams will select 
monitoring indicators as part of their work in Step 1: Get Organized. 

Early Warning Implementation Key Terms 

Having a common language for talking about, understanding, and using early warning data to support 
student needs may help your team use early warning data more effectively. Throughout the guide there 
are several key terms that are used, which are described in Box 1. 
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Box 1. Early Warning Implementation Key Terms 


Early warning data: Early warning data is a general term that refers to data used to predict, diagnose, 
and monitor student risk for not meeting academic milestones. Early warning data refers to EWIS 
student risk levels and other data found in the EWIS reports, as well as local data sources which 
include monitoring indicators and diagnostic data. 

Early warning implementation: The multi-step process that districts and school can follow to 
effectively use early warning data to identify at-risk students, explore underlying causes for risk, 
provide interventions and supports, monitor student progress, and make schoolwide improvements 
to lower risk for all students. This multi-step process is a key foundation for the Massachusetts Tiered 
System of Support (MTSS). 

Massachusetts Early Warning Indicator System (EWIS): The early warning system created by the 
Massachusetts Department of Elementary and Secondary Education that identifies students who are 
at risk of missing key academic milestones in grades 1-12. The four key academic milestones are 
reading by the end of third grade, middle school ready, passing all grade 9 courses, and high school 
graduation. The EWIS provides student risk levels at the beginning of each school year. Student-level 
and aggregate EWIS information is available through five reports in Edwin Analytics. 

EWIS Student Risk Level: Each student who attended a Massachusetts public school in the year prior 
is assigned a level of risk of low, moderate, or high. A student's risk level indicates how likely it is that 
the student will miss the key academic milestone for their grade level, such as passing all their 9 th 
grade courses or graduating from high school. EWIS student risk levels are based on statistical 
formulas that are applied to multiple sources of readily available data from the prior year. Students 
who are assigned a student risk level of high are predicted to be most off-track for reaching the next 
key academic milestone. Conversely, students who are assigned a risk level of low, are predicted to 
be on-track for reaching the next key academic milestone. 

Local Data: Schools and districts can complement their use of EWIS with local data sources. There are 
two types of local data sources: monitoring indicators and diagnostic data. 

■ Monitoring indicators. Monitoring indicators rely on data that are available during the school 
year. Data sources such as attendance, academic performance, and behavior have been 
identified in research as strong predictors of student risk for missing key academic milestones. 
Monitoring indicators complement the use of EWIS student risk levels. After receiving EWIS 
student risk levels at the start of the school year, Massachusetts' districts can use monitoring 
indicators to flag students who are at risk and track student progress at multiple times 
throughout a single school year. Monitoring indicators can either be research-based or locally 
determined. 

■ Diagnostic Data. Additional data sources, such as demographic data, benchmark data, and state- 
level student data available through Edwin, is used by schools and districts in combination with 
EWIS student risk levels and monitoring indicators. These supplemental data sources can help 
schools and districts better understand underlying causes of student risk, and track student 
progress in interventions. 
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Section 2: The Six-Step Early Warning Implementation Process 

To maximize the value of early warning data, educators can use early warning data within the context of 
a multistep, data-driven implementation process. This allows educators to use early warning data to 
match interventions and supports to student needs, monitor student progress to ensure that students 
stay on track and are responding to interventions, and strengthen schools along the way. 


The Early Warning Implementation Theory of Action 


Exhibit 3 illustrates the early warning implementation theory of action and how it is intended to support 
student success. The implementation process is supported by three components (as shown in the left 
column of Exhibit 3): (1) EWIS, monitoring indicators, and diagnostic data; (2) the Early Warning 
Implementation Guide and other training, resources and supports; and (3) the district and/or school 
team or staff dedicated to the use of these data. Exhibit 3 illustrates how these components contribute 
to improved student outcomes. 


Exhibit 3. Theory of Action for Early Warning Implementation 


COMPONENTS 


I EWIS, I 
I monitoring I 
i indicators, and I 

I diagnostic data j 

i i 

Early Warning ■ 
: Implementation 
: Guide and other : 
l resources, ; 
l training, and ] 

I supports j 

District and/or : 

school-based . 
: team focused on ; 
, using early | 

j warning data to j 
, support students j 


INTERMEDIATE OUTCOMES OUTCOMES 




D 

Improved 

likelihood 

for 

students in 
need to 
receive 
apt 

supports 
and get 
back on 
track 


E 

Improved 
student 
success in 
reaching 
desired 
academic 
milestones 


It is expected that the combined use of these components will (1) improve educators' use of data to 
identify students who are at risk for not meeting academic milestones, (2) improve educators' ability to 
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efficiently and effectively match interventions and supports to meet student needs and make broader 
schoolwide improvements to reduce risk for all students, and (3) improve educators' ability to monitor 
student progress over time. The arrow at the bottom of the illustration represents an expected cycle, 
where progress through the pathway from A to C yields continued improvements in these intermediate 
outcomes. The intermediate outcomes, in turn, are expected to promote critical student outcomes, 
particularly students' likelihood of getting back on track by meeting EWIS benchmarks (D) and, 
ultimately improved student success in meeting academic milestones (E). 


Getting Ready for Early Warning Implementation: Team Exercise 

Before learning more about the six-step early warning implementation process, staff may want to 
become more familiar with early warning data, and consider how its use will best fit with what they are 
already doing in their district or school. The following two activities may help you get ready to use early 
warning data effectively as you work to support students and drive improvements. 

Box 2. Getting Ready for Early Warning Implementation: Team Exercise 


Explore EWIS: Take a spin through your current EWIS data and reports in Edwin Analytics. Start by 
running the EWIS District View Report (EW301). What did you notice about the extent of risk for your 
district or school? Which school or grade had the highest proportion of students at risk? The lowest? 
Did you see anything in your EWIS data that was surprising? What new questions did you have after 
reviewing your EWIS data? What other uses can you foresee for your EWIS data? Who might want to 
use it-why and when? 

Where does early warning implementation fit? Consider the various ways that your district or school 
is already using student data? How is data used to determine Tier 2 and Tier 3 supports and 
interventions and/or to evaluate their effectiveness? Which individual staff members, committees, 
or teams use data to identify and support students? Which committees or teams use (or could use) 
data to inform decisions about school improvement efforts? Considering your existing data uses, 
committees, teams, and improvement initiatives— where and how might early warning 
implementation fit? Would early warning data add value or focus to these existing efforts? Are you 
better off convening a new and separate group to specifically focus on using early warning data? 
What are your next steps for determining who, when, and how you will use early warning data in 
your school or district? 


Overview of the Six-Step Early Warning Implementation Cycle 


This guide outlines a six-step EWIS implementation process adapted from AIR's National High School 
Center publications on early warning system implementation. 4 The process draws on research on data 
use in districts and schools 5 in the context of implementing an early warning system, such as EWIS, and 
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is similar to a cycle of inquiry, such as the one outlined in the Massachusetts District Data Team Toolkit 

( http://www.doe.mass.edu/apa/ucd/ddtt/Knowledge.pdf) . 

In this guide, the six steps are presented in a sequence, but they are intended to be cyclical. Exhibit 4 
illustrates the cycle. At the core of this data-driven, decision-making process, the steps focus users on 
using EWIS data in combination with local data, including monitoring indicators (e.g., attendance, 
behavior, course performance) to identify which students are showing signs of risk of missing key 
academic milestones and diagnostic data which helps teams know how to take action. Teams use data 
to identify, diagnose, assign, provide, and monitor interventions and supports for individuals and groups of 
students as well as make schoolwide improvements to lower the risk for all students. The six steps guide 
users through a straightforward process that supports informed decisions based on these data and other 
information. 

Exhibit 4. Six-Step Early Warning Implementation Process 



Adapted with permission from the Early Warning Intervention Monitoring System ™ implementation guides for the middle grades and high school, 
developed by the National High School Center. (See Endnote 1.) Copyright © 2013 American Institutes for Research. All rights reserved. 
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The remainder of Section 2 of this guide outlines the tasks and the processes for implementing each 
step in the cycle. The guide is intended to be a flexible tool to help educators customize their use of 
early warning data to fit their school's individual needs, interests, and resources. No two schools are 
alike, so early warning implementation may vary across districts and even among schools within a 
district. For example, some schools or districts may find that they have a high percentage of students 
who are identified as being at risk. For these schools and districts, their focus may be primarily on 
schoolwide changes in addition to interventions and supports for individual students and groups of 
students identified as being in need. Another district may find that they have very few students who are 
identified as being at risk. This district may focus its efforts primarily on providing support and 
interventions to meet the needs of individual students. 


Implementation Timeframe 

The early warning implementation process is carried out during the course of a school year and is 
aligned with the academic calendar. Specific steps are undertaken at defined periods of the year, many 
in a recurring or a continuous manner. As the timeline shown in Exhibit 5 illustrates, teams begin to 
organize their work prior to the start of the school year (Step 1). Typically, teams will implement Steps 2, 
3, and 4 during the first six weeks of the school year. After student interventions and supports have 
been assigned (Step 4), teams periodically review monitoring indicators to ensure that support 
strategies and interventions are responsive to student needs and use monitoring indicators to ensure 
that all students remain on-track (Step 5). As shown, it is recommended that teams monitor students at 
the end of each grading period. In the spring, teams use monitoring indicators data to inform the 
planning of summer supports for students. Finally, at the conclusion of the school year, teams refine 
their approach to using early warning data (Step 6). 


Implementing the Six-Step Process 


The following subsections offer detailed, step-by-step information to guide implementation efforts 
through the six-step implementation cycle. Each step includes the following: 

■ What you need to implement the step 


■ Description of the step 

■ Anticipated outcomes 

■ Notes from the Field, anecdotes and concrete examples for early warning 
implementation in Massachusetts 



■ Build on your Foundation, ideas for how your team might build upon 
current systems and practices 

■ School and district roles 

■ Guiding questions 



■ Tools, such as step implementation checklists, worksheets, and action planning tools 
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Exhibit 5. Early Warning Implementation Timeline 
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Step 1: Get Organized 


What You Need for Step 1 

■ Interest and commitment from appropriate administrators and staff from the school and/or the 
district who are able to meet regularly as a team to use early warning data to identify and support 
students in need 

■ Sufficient time to meet regularly throughout the school year (at a minimum, a monthly meeting 
lasting one to two hours) 

■ Access to EWIS data and reports within Edwin Analytics 

Description of Step 1 

There are three critical parts to getting organized. 

1. Convening a team or group (existing or new) that will be responsible for reviewing early warning 
data and making decisions about supporting students in need. 

2. Devising the plan for the year, including your teams' goals, meeting times and frequency, data 
import schedule, and how you will communicate and coordinate with others. 

3. Getting ready to use early warning data including accessing EWIS student risk levels in Edwin 
Analytics and identifying which data your team will use to monitor students over the course of 
the school year. 

A. Convening the Team(s) 

A diverse, well-informed team is essential to the effective use 
of early warning data for identifying and supporting students. 

The team designated as responsible for using EWIS data and 
other early warning data, such as monitoring flags, to identify 
and support students may be established as a new team. 

Alternatively, a school or district may choose to build upon 
the work of an existing team (e.g., the school data team, the 
school improvement team, the tiered system of support team, 
and/or the student support team). Regardless of whether the 
team is newly formed or integrated with an existing team, the 
team should include a broad array of staff within the school 
and/or district and should be prepared to dedicate sufficient 
meeting time to reviewing, interpreting, and acting on student 
early warning data. To help you get organized, refer to Tool 4 
in Step 1. 



Build Off Your Foundation 

1 How do you currently identify 
and support students in need? 

1 How do you currently monitor 
student progress? 

1 How can you integrate the use of 
EWIS and other early warning 
data into the work of your current 
teams/committees? 

'Are there ways you can merge 
EWIS data and other early 
warning data with your current 
data systems? 


STEP 1 
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Anticipated Outcomes for Convening the Team 

■ The identification of an existing team(s), or the 
convening of a new team(s) at the school and/or district 
level 

■ The selection of team members who have a diverse set 
of knowledge and skills and are committed to the use of 
early warning data for identifying and supporting 
students 

■ Clarification of roles and responsibilities of the team, 
including the identification of one or more individuals 
to serve as chair and co-chair, logistical support, and a 
data specialist (who is responsible for importing and 
managing early warning data) 

Structuring the Team 

Early warning implementation requires the engagement of 
stakeholders at multiple levels. Although much of the action 
of identifying underlying causes and supporting students in need occurs within school-level teams, the 
district has an important role to play. Teams may vary in structure, number, and composition, 
depending on the size of the district. Large districts with many schools may have one school-based team 
at each school as well as a district-level team with both school and district representation. Smaller 
districts may have one school-based team in each school with both school and district representation. 
For example, in one school, the school-based team may be composed of a content area teacher (e.g., 
reading or mathematics), a school counselor, a technical/data person, and a principal or an assistant 
principal. The school team may meet monthly, and the district administrator may attend every other 
month. Such a scenario streamlines communications between the district and the school regarding 
implementation and allows team members to identify implementation challenges that are best resolved 
or addressed by the district. 



Notes from the Field 


To have the K-12 lens on our 
students and such a variety of 
different people talking about 
student needs really broke down 
silos. Nothing happens in a 
bubble - so to get all the 
different perspectives focused 
on one issue was huge. 




The following briefly describes the role of both the school and the district in the implementation process 
associated with using early warning data. 


School Role. The primary role of each school is to use EWIS and other early warning data to identify and 
support students within the school, and to help these students meet desired academic milestones. 
Within each school, the team will draw on available information, coordinate and collaborate with 
existing committees and groups, and bring together additional educators and specialists to explore 
schoolwide changes. The team, in consultation with other stakeholders, including students and family 
members, can match individual students to appropriate interventions and supports and develop 
programs and supports to meet the needs of groups of students. Each school also may want to be in 
regular communication with its feeder school(s) to facilitate transitions and assist district administrators 
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in coordinating efforts across the district. The intensity and the frequency of activities and 
communication associated with the use of early warning data will vary by school but are likely to be 
greatest in the fall. 


District Role. The role of the district is to support efforts within schools to use and respond to early 
warning data, facilitate communication and coordination across schools (particularly for school 
transitions), and identify and address system-wide concerns. For example, a district might observe that 
many students coming from specific elementary schools are more likely to be identified as being at high 
or moderate risk after the first year of middle school. The district may want to examine the underlying 
causes for this pattern. Perhaps one feeder elementary school is not preparing its students for middle 
school in the same way that other elementary schools are. Or perhaps the particular needs of these 
students are addressed in elementary school but are not sufficiently addressed in the first year of 
middle school. In addition, district administrators play a key role in establishing a mission, goals, and a 
direction for the early warning work, and are well-placed to communicate the importance of EWIS and 
other early warning data. The district also can play a critical role in ensuring that school teams have 
access to data, resources, training, and other supports necessary for effective implementation. 

Team Composition 

Strong teams include a diverse, broad, and layered membership that represents a range of skills and 
expertise (e.g., leadership, technical expertise, instructional expertise, social/emotional, and mental 
health expertise.) It is suggested that the team(s) consist of personnel who have both the authority to 
make decisions about interventions and supports, staff, and students, and familiarity with a diverse 
array of students, including students with disabilities and students who are English language learners. In 
addition, it is critical to have at least one individual with the skills to access and analyze EWIS data (in 
Edwin Analytics) and the ability to import and analyze additional early warning data from other student 
data. Over the long term, membership on the team will likely evolve and shift. Although it is good to 
rotate and engage more staff in the process over time, it is helpful if some individuals continue to serve 
on the team across multiple years to ensure continuity and consistency. 


Implementation Tip : 

Potential Team Members 

School-based team (may include district, school and community reps): 

• School principal or assistant principal 

• Representatives from feeder schools (i.e., elementary, middle) 

• District office representative 

• Special education teachers 

• English language learner (ELL) instructors 

• School counselors 

• School psychologist or social worker 
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• Content area teachers 

• Data specialist (or someone proficient in using the school's student data systems) 

• Afterschool director 

District-based team (may include district, school and community reps): 

• Superintendent or assistant superintendent 

• Data specialists 

• Testing/assessment/accountability administrators 

• Curriculum and instruction representatives 

• Pupil personnel/ student support administrators 

• Special education administrators 

• ELL administrators 

• Principal/administrator or another administrator from each school 

• School counselors and/or social workers 

• Classroom teacher representatives 

• Others (e.g., members of other district and school teams and committees) 


Team Roles and Responsibilities 

Whether the school or district team is integrating early warning implementation work into an existing 
committee, team, or group, or forming a brand new team, it will be important for the team to function 
effectively, efficiently, and professionally. One of the most important initial steps is to determine the roles 
and the responsibilities of the team members. It is suggested that the team designate a chair and a co- 
chair to provide leadership, facilitate meetings, and serve as the primary liaisons to stakeholders outside 
the group. Many teams also may want to designate an individual to provide logistical and administrative 
support to the team. Finally, most teams dedicated to using early warning data will want to designate at 
least one individual, either on the team or in support of the team, who will act as the primary data 
specialist. This should be someone with proficiency in using and linking student data systems with other 
databases, and someone who has sufficient time available to manage the importing of student data, and 
production of early warning data reports. 

Guiding Questions for Convening the Team 

■ Will the team be a newly established group or integrated into an existing committee, team, or 
group? 

■ Will there be a separate district-level team or only district representation on school teams? 

■ What types of staff need to be represented on the team (e.g., district administrators, counselors, 
teachers, and/or data specialist)? 
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■ What types of knowledge and expertise do team members need to have (e.g., knowledge of student 
needs, diverse student populations, awareness of family and cultural factors, data analysis 
strategies, existing supports, and/or dropout prevention programs)? 


■ How might the composition of the team differ across school levels within a district (e.g., should 
membership be different at the elementary versus middle versus high school levels)? 

■ Who will serve in key roles on the team (e.g., chair, administrator/manager, note taker, data 
specialist)? 

B. Devising the Plan 

Effective teams articulate clear goals and procedures for effective team functioning and meetings, 
establish regular meeting times, and devise a plan for communicating and coordinating with others. 

Anticipated Outcomes for Devising the Plan 

■ Clarity and consensus regarding how the team will conduct its work (e.g. clear, written goals and 
objectives, and plans for how the group will conduct effective meetings) 

■ Schedule of regular meeting times to review, interpret, and act on early warning data throughout 
the school year. 

■ Clear plan for how the team will communicate and coordinate with others outside the team, 
including how the team will protect the confidentiality of student data. 

Team Goals and Functioning 

The team(s) dedicated to the use of student early warning data will meet to review student data, 
explore the underlying causes of risk, and align students in need with appropriate interventions and 
supports. The team also may want to explore group or schoolwide changes that may meet the needs of 
a broader range of students. Throughout the year, continuous monitoring of students who display risk 
flags will improve the team's ability to assign appropriate interventions or supports and will allow 
midcourse corrections if a particular student or a group of students does not respond to an intervention 
or support. The team can facilitate its functioning and enhance effective communication throughout the 
school by doing the following: 

■ Setting clear written goals and objectives for the work of the team. This will be particularly 
important in the first year of implementation. Many teams have found it helpful to narrow the focus 
and expectations for their work during the first year to allow them to refine team functioning and 
effective use of early warning data before attempting schoolwide implementation. 

■ Establishing rules and procedures for protecting the confidentiality of student data. Teams will 
need to determine who will have access to sensitive student data and include written instructions 
and provisions for protecting confidentiality on reports and other documents containing sensitive 
information. 
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■ Communicating with individuals and groups outside 
the team. Teams may want to share information with a 
range of other individuals and groups in their efforts to 
support students, including other school and district 
committees and teams, teachers and specialists, and 
social service agencies, and community based 
organizations such as afterschool providers, who are 
already, or could be providing services to students 
identified as at risk. For example, teams may want to 
communicate regularly with school and district 
leadership about students in need, intervention or 
support effectiveness, and team-identified needs for 
support. Teams also may want to provide teachers with 
regular updates about students in their classes who are 
displaying risk flags, as well as information about 
supports available to teachers in working with these 
students. Teams may want to collaborate with 
afterschool providers to align in and out of school 
support for at-risk students. Teams may want to 
establish procedures and rules regarding how, when, and with whom they will communicate 
information associated with their activities and findings and how they will protect the confidentiality 
of student information being shared. 

■ Communicating and collaborating with students and families. In some cases, the team may want to 
communicate with students and families about their concerns, or convey these concerns through a 
teacher. Collaborating with students or families may help to ensure that students are adequately 
supported and that all interested parties have a chance to help the student get back on track. Of 
critical note, the team should share concerns about students with sensitivity; sharing specific data 
about EWIS risk levels is not advised. It is important to ensure that any shared information is used to 
prompt action and support, not give labels that carry stigma. 

For example, a student with an EWIS student risk level of "high" may have consistently poor 
attendance in the new school year. Rather than sharing with the student and the family that the 
student is "high risk," a more productive and specific approach is to inform the student and the 
family that the poor attendance is likely contributing to being off track for meeting an important 
academic milestone (e.g., passing all grade 9 courses). This approach allows for a discussion for 
possible reasons for poor attendance that may be addressed. 

Team Meetings 

Strong teams set aside adequate time to accomplish their work and establish a consistent meeting 
schedule, space, and set of procedures. The following are a few strategies to support effective and 
efficient team meetings. 



Notes from the Field 


Our EWIS school team 
shared EWIS data with 
guidance and school 
adjustment counselors. These 
counselors were able to use 
EWIS data when developing 
Individualized Student 
Support Plans as part of their 
Response to Intervention 
(RTI) work. 
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■ Meet regularly to monitor student progress. The team will want to meet regularly (within the first 
30 days of school and at least at the end of each grading period) to monitor progress as it strives to 
improve the educational outcomes for students. Many teams find that they need to meet more 
frequently in the fall in order to implement the initial steps in the implementation cycle. This portion 
of the cycle ensures that students identified as at-risk at the start of school can be assigned to 
appropriate interventions. It will be important to schedule meetings around the availability of 
student data, allowing sufficient time for these data to be entered and analyzed prior to the team's 
meeting date. 

■ Conduct well-organized and documented team meetings. Clear agendas are important to ensure 
that each meeting has goals and a focus. At least some agenda items will likely be routine, such as a 
review of student data, actions taken for individual students or groups of students, a review of 
previous meeting action items (ongoing or completed), and communication with staff and leadership. 
Meeting minutes or notes can help teams to document decisions and organize any additional 
information that has been gathered and/or analyzed by the team, and help them keep track of action 
items and assigned tasks. 

■ Establish procedures for making team decisions and assigning work. Team members have many 
responsibilities, and limited time is often an impediment to reaching team goals. Having procedures 
in place for making team decisions fairly and efficiently will help meetings to go more smoothly. In 

addition, many teams find that having clear roles, 
responsibilities, and procedures for delegating tasks 
enhances the team's effectiveness. 

■ Implement the steps outlined in the implementation 
cycle. Once organized, the work of the team will focus on 
implementing the steps outlined in the implementation 
cycle (i.e., generating early warning data reports, reviewing 
and interpreting early warning data, assigning interventions 
and supports, and monitoring student progress). Team 
members should be knowledgeable about this cycle and 
receive training on how to implement key steps associated 
with the use of student early warning data. 

Guiding Questions for Devising the Plan 

■ What are the team's priorities, principles, and 
expectations regarding how the team will conduct its work 
(i.e., what is the group's process)? 

■ How frequently and on what specific dates should the 

team meet? 

■ What does the school-level team hope to accomplish? What does the district-level team hope to 
accomplish? 



Notes from the Field 


We realized that it doesn’t need 
to be another meeting! The 
EWIS work can be incorporated 
into an existing team rather than 
trying to create a new team of 
professionals who are already 
taxed and pulled in a variety of 
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■ How will the team members ensure they have sufficient time and resources to meet and conduct 

their work outside EWIS meetings? 

■ What types of technical support and/or professional development are needed to train and support 

the team(s) at the school and district levels)? 

■ How and with whom will the team communicate about the team's mission, activities, and progress? 

C. Getting Ready to Use Early Warning Data 

An early warning system uses readily available student data to identify students who are at risk of not 
meeting desired academic milestones. To launch your early warning implementation process you will 
need to identify and access the student data the team will use to identify, support, and monitor the 
progress of students in need of support. 

Anticipated Outcomes for Getting Ready to Use Early Warning Data 

■ Access EWIS data and reports 

• At least one member of each team will have proficiency in: accessing EWIS data (through Edwin 
Analytics); using the various reporting options for examining student risk information; and 
exporting EWIS student-level data (EW601) to Excel for further analysis. 

• All members of each team will understand the four academic milestones in EWIS, be familiar 
with the types of data underlying the risk levels, and have the ability to understand the various 
EWIS reports. 

■ Preparing for the monitoring of student risk 

• The team will have identified which student data 
sources will be used to monitor student risk 
throughout the year. 

• The team will have established thresholds that will 
be used to "flag" students as being at risk based on 
these data sources. 

• The team will have selected (or set up) the data 
system or tool that will be used for monitoring 
student risk flags. 

• At least one member of the team, or someone 
supporting the team, will have proficiency in 
updating monitoring indicator data and generating 
reports using the monitoring indicators database 
system or tool. 



Notes from the Field 


Prior to EWIS, we relied only on 
guidance counselors and 
teachers to identify students 
having academic difficulty and 
bring them to the Instructional 
Support Team. The EWIS data 
has inspired us to develop our 
own academic warning system, 
which expanded upon the EWIS 
tool by incorporating additional 
data points. This system allows 
us to systematically combine 
data on all students, including 
professional perspectives. 
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Accessing EWIS Data and Reports 


EWIS data reports in Edwin Analytics provide information on students' risk-level status at the beginning 
of each school. The team will need to access these data to understand the extent of student risk at the 
beginning of the school year. EWIS data and reports provide information to districts and schools on 
students' risk levels based on data from the prior year. ESE applies risk formulas to existing data for all 
Massachusetts public school districts, resulting in an assigned level of risk for every student. Within 
Edwin Analytics, districts and schools can view their EWIS data through five report options: 

■ District View (EW301) 

■ Graphical View (EW302) 

■ Risk Level Indicator Analysis (EW317) 

■ Subgroup Analysis (EW318) 

■ Student List (EW601) 

Within each option, districts and schools can filter the data in numerous ways (e.g., select specific schools, 
grade levels, and/or student subgroups) to gain varying perspectives on their data. These reports allow 
districts and schools to understand the scope, the depth, and the nature of student risk levels. EWIS 
data are accessed from Edwin Analytics, which is available through ESE's Security Portal. The Security 
Portal can be found in the drop-down menu on the upper-right side of the ESE website 
( http://www.doe.mass.edu/ ) or directly at https://gateway.edu.state.ma.us/ . For more information 
about accessing Edwin Analytics and the EWIS data and reports, please see Appendix B or 
http://www.doe.mass.edu/edwin/analytics/ . 

Exporting EWIS Data 

All reports in Edwin Analytics may be exported to PDF or Excel formats, which may be helpful for sharing 
and further analysis. Some districts may want to expand their capacity to examine EWIS data beyond the 
reporting options offered from Edwin Analytics and therefore may choose to export their student-level 
EWIS data (Edwin Analytics report EW601) into Excel. After the EWIS data are exported, districts and 
schools can combine these student-level data (which includes all of the data points that were used to 
assign a student to a particular risk level) with other sources of district and/or school data and/or 
generate a wider range of data reports. 

Preparing to Monitor Students 

One of the keys for using early warning data is the frequent monitoring of student progress using readily 
available data. Monitoring indicators can be used during the school year to flag students in need, and 
track progress. Data such as attendance, academic performance, and behavior have been identified in 
research as strong monitoring indicators. Monitoring indicators can complement the use of EWIS data 
by enabling teams to flag students who become or continue to be at risk during the school year. 
Monitoring indicators can also be used to track students' responses to interventions or supports at 
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multiple times throughout a single school year and thereby assist teams in making decisions regarding 
interventions and supports. Diagnostic data, another local data source, such as benchmark assessment 
data, student focus groups, and staff professional opinions, can be used in conjunction with research- 
based monitoring indicators. Diagnostic data is particularly helpful in exploring the underlying causes for 
student risk, and tracking student progress in interventions. 


Getting Ready to Monitor Students and Interventions 

During Step 1, teams prepare to use early warning data, 
including monitoring indicator data before or during the 
early part of the school year. These steps need to be 
completed prior to monitoring students and interventions: 

■ Select which student data sources the team will use to 
monitor student risk throughout the year. 

■ Establish thresholds to "flag" students as being at risk 
using these data sources (e.g., students who miss 10 percent or more of school days). 

■ Establish a system or tool your team will use to monitor students. 

■ If using a data system or tool, make sure that at least one member of the team has proficiency in 
importing, updating, and generating reports using the database system or tool that contains student 
data. 

■ If using a data system or tool, establish a data import and review schedule for analyzing data and 
reports with up-to-date student monitoring indicators data throughout the school year. 

■ Verify that all members of the team understand the data sources for monitoring student progress and 
the various monitoring indicator reports. 

Selecting Monitoring Indicators 

Teams can select from one or more monitoring indicators. Decisions about monitoring indicators should 
be grounded in research or local validation of data. A monitoring indicator draws from readily available 
data that can be observed at many points during the school year. As shown in Tool 3 below, monitoring 
indicators are data sources with identified thresholds, defined in research or locally validated whereby 
users can observe whether a student is flagged because he or she met or exceeded this predetermined 
threshold. For example, if the data source is attendance and the threshold for flagging students is 
missing 10 percent of school days, then any student who misses 10 percent or more of school days 
would be flagged as at-risk. The frequency with which flags are examined is dependent on the schedule 
set by the team. 

Selection Considerations. Before, or at the start of the school year, districts and schools must select the 
specific monitoring indicators they will use to track student progress throughout the year. The guiding 
principles for selecting monitoring indicators are as follows: 


Implementation Tip : 

Go Slow 

Proceed slowly when generating data 
reports to avoid an overload of 
information. Remember, data are useful 
only when team members can understand 
and make use of the information provided. 
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(1) Indicators are research based or locally validated. 

(2) The indicator is based on readily available data that can be updated and accessed several times 
throughout the school year. 

(3) More than one, but no more than four indicators per grade level, are used by the team to monitor 
student risk throughout the year. Remember, monitoring indicators are used to identify students 
who are displaying symptoms of need, not to diagnose 
need. 


After considering the list of possible monitoring indicators in 
Tool 3, some districts and schools may also want (or need) 
to use other local data sources to flag and track student 
progress using data that is more closely aligned with the 
needs and practices of their district, schools, and/or 
students. For example, if a district or a school uses a 
commercially available benchmark assessment tool, to 
determine students' academic progress in reading, then this 
information may be a source of data that can be used to 
identify at-risk students and track student progress during 
the year. Some schools and districts may want to select data 
sources and establish performance thresholds more closely 
associated with particular supports to monitor student 
progress in these interventions. 


Implementation Tip : 

Be Discerning in Your Selection of 
Monitoring Indicators 

Each grade level should only use 2-4 
monitoring indicators since these 
indicators are used just to flag students 
throughout the school year (not to 
diagnose their needs). If you are new to 
using monitoring indicators, you may want 
to start with only 1 or 2 monitoring 
indicators per grade level. Using 
attendance as a monitoring indicator is 
often a good place to start. Attendance can 
be a powerful monitoring indicator and 
easily accessible. 



Notes from the Field 


For monitoring students, we 
decided to use attendance, 
grades, “Orleans-Hannah” and 
“Galileo’’ data, as well as our 
reading inventory from the 
beginning of the year. 





Data Systems for Monitoring Students and Interventions 

It is recommended that the team identify or establish a 
system before, or early on in the school year, to manage 
the data that will be used to track monitoring indicators. 
The team has various options for managing these data. For 
instance, districts and schools may wish to use their own 
student information systems for monitoring students. 
Districts and schools may choose to use an existing system 
for monitoring students, such as AIR's National High School 
Center EWS High School Tool or EWS Middle School Tool 6 
which is available for free, or other commercially available 
monitoring indicator tools. Note that in these cases there 
must be a person who is dedicated to importing student 
data into the tool several times over the course of the 
school year. In either case, the team must ensure that it 
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has access to needed student data throughout the year, and has a schedule for data access. 


Guiding Questions for Getting Ready to Use Early Warning Data 

■ EWIS Data and Reports 

• Do all team members understand the academic milestones for the age groups on which the 
team is focused? Do all team members have a general understanding of how the risk levels were 
determined? 

• Is there someone on the team who knows how to access EWIS information and generate a range 
of reports? Who will be responsible for generating the EWIS reports? 

• Does the team have an interest in generating more reports or combining EWIS data with other 
student data? Does the team have someone who knows how to export EWIS data and merge 
these data with other data systems? 

• With whom will EWIS data be shared? How will student confidentiality be protected, as required 
by district policies? Who will provide reports to other stakeholders? 

■ Monitoring Indicators 

• Which data sources make the most sense for our school to use to monitor its students? Will our 
team use the research-based indicators from Tool 3? Should we use our own locally validated 
indicators? 

• What thresholds will the team set to determine when students should be "flagged" as at risk? 
Are these selected data sources and thresholds research-based or can they be validated locally? 

• Does the team want to use any diagnostic data sources to track student progress in assigned 
interventions, such as available benchmark assessment data? 

• What data systems or procedures does the school already have in place for tracking student 
progress? Does the team want to use its own data system or import data into a commercially 
available monitoring indicator tool? How frequently will the team examine risk flags based on 
these data sources? 

• Who will be responsible for obtaining the data to monitor students? Who will generate data 
reports showing students flagged as at-risk, for the team? 

• With whom will these data be shared? How will student confidentiality be protected, as 
required by district policies? Who will provide reports to other stakeholders? 

District Role in Step 1 

The role of the district and/or the district team is to identify system-wide concerns and develop and 
recommend districtwide changes that address such concerns. District administrators also play a key role 
in establishing a mission, goals, and direction for the early warning work and communicating the 
importance of early warning data, including EWIS, within and across schools. District administrators can 
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participate in the work of school teams, provide professional development, and support and oversee 
school-level efforts. To maximize the impact of the district role, we recommend the following: 


■ Engage in school-level meetings or routinely communicate with teams to increase attention to 
efforts and signify the importance of using early warning data. For example, on one team, a district 
administrator might participate in a team meeting at designated times during the school year. This 
allows the team to share information about students in the school and identify challenges and needs 
based on their use of early warning data. The district administrator benefits by having the 
opportunity to hear about challenges facing the team and consider ways in which the district may 
support this work. For example, the district administrator might assign a data specialist to provide the 
team with information in a particular, more user-friendly manner, or connect the team with a school 
that has had success in supporting students. 

■ Provide professional development to team members on using EWIS or other student early data. This 
training can enhance the work of the team and decrease variation in the quality of the team's work. 
For example, a district may offer professional development to new school-level team members at 
the end of each school year or during the summer to prepare them for the work ahead. 

■ Arrange for school-level team members to have release time, coverage, or other supports to enable 
them to participate in meetings and related work. 

■ Ensure that school-level teams have been prepared for and have access to student-level data (e.g., 
knowledge of confidentiality procedures/guidelines, passwords, and instructions for accessing). 

■ Monitor school efforts to use early warning data throughout the school year and over the course of 
multiple school years to ensure that schools are improving outcomes for students. This allows the 
identification of promising practices and areas of need in the district as a whole. Districts may 
monitor and track early data use and implementation by participating in meetings or requesting 
presentations on school-level work. This gives the district the opportunity to identify common needs 
across schools and develop districtwide strategies for allocating resources or sharing practices. 

■ Ensure that teams have access to EWIS data and reports in Edwin Analytics. 

■ Ensure that at least one or more members have been trained in using Edwin Analytics. 

■ Provide technical support (either directly or through a third party) to team members in accessing and 
modifying the current data system to track specified indicators or in learning how to use existing 
monitoring indicator tools (if preferred). 

■ Work with school-level teams on selecting the student data sources and thresholds that will be used to 
monitor student progress that allow districts to have the desired consistency in data collection across 
schools. 

If there is a district-level team in place, this team may meet less frequently than school-based teams 

(e.g., two to four times per year) to discuss: persistent problems and challenges; resources and 

strategies for supporting students; and systemic, organizational, and policy changes that may be 

needed. School-level representatives can help the district team develop new districtwide strategies for 


Early Warning Implementation Guide 


Page 25 




students in need (e.g., new policies, schedules, partnerships, curriculum, or training for teachers and 
students). 


Early Warning Implementation Guide 


Page 26 




Tool 1 - Step 1: Checklist 


The Team 

□ Determine team structure (e.g., new team or existing team? School level, district level, 
combined, or both?) 

□ Determine team composition (who will be part of the team?) 

□ Determine team roles and responsibilities (e.g., leader, data designee, logistics) 

The Plan 

□ Determine rules and expectations related to the team's group process (decision- 
making, conflict resolution, and ground rules) 

□ Communicate with the district to clarify goals and expectations (district and school) 

□ Set team goals, objectives, timeline and tasks 

□ Construct a team communication plan 

□ Set team meeting schedule (timing, frequency, and length) 

The Data 

□ Confirm EWIS data access 

□ Select locally meaningful monitoring indicators for observing changes in student risk 
over the course of the school year 

O Confirm that monitoring indicator data sources are accessible and readily 
available 

□ Assign responsibility for providing the team with monitoring indicator data and reports 
in preparation for team meetings 

□ Identify other local data sources to be used in diagnosing student needs and tracking 
progress in interventions 
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Tool 2 - Step 1: Where Are You Now? 


Directions: Form small groups of 3-4 people. Share with each other where you are now with your use of 
data and your current approaches for identifying and supporting students in need. You may discuss all 
the questions, or focus on one or two in more depth. 

Discussion Questions: 

1. How do you currently identify and support students in need? 

■ What types of data do you use to identify students in need? 

■ Do you use EWIS data to identify students in need? If yes, how do you use these data? 

■ Do you use other student data? How do you use these data? 

* Who is responsible for identifying students in need and ensuring that support is provided to 
these students? Is this the role of one individual, several individuals, or a team? 


2. How do you currently monitor student progress? 

■ Do you examine data to identify students in need during the school year? 

■ How frequently do you monitor the progress of students who are identified as in-need? 

■ What database are you currently using to organize these data? 


3. How can you integrate the use of early warning data, including EWIS, with your current 

practices? 

■ Which aspects of your current approaches for identifying and supporting students in need 
do you feel are adequate or good? 

■ Are there changes or modifications you would like to make to your strategies for identifying 
and supporting students in need? What are they? 

■ How do you think your use of early warning data and the early warning implementation 
process can be integrated with or strengthen your current practices? 
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Tool 3 - Step 1: Potential Monitoring Indicators by Grade Level 

Directions: The table below contains possible monitoring indicators to use throughout the school year for each grade level. Each grade level 
includes attendance, behavior, and academic performance potential monitoring indicators flags students that are on or off track. Some of these 
monitoring indicators are more strongly supported by research (marked in the “Research Supported" column), while others are commonly used 
but need to be locally determined and validated to establish an appropriate threshold. In the "Consider Using?" column, check of which 


monitoring indicators your school or district could potentially use and note the data source and possible threshold for the monitoring indicator. 
If you are new to using monitoring indicators, you may want to start with only 1 or 2 monitoring indicators per grade level. 


Academic 

milestone 

Grade 

Potential 

Monitoring 

Indicator 

Threshold 

Research 

Supported 

Locally 

Determined 

Consider 

Using? 

Notes - existing data source, 
possible threshold, etc. 

Early Elementary 



Proficient or higher 
on grade 3 ELA State Assessment 

1 

Attendance 

Locally defined 


X 



Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 




2 

Attendance 

Locally defined 


X 



Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 




3 

Attendance 

Locally defined 


X 



Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 





Additional notes about possible monitoring indicators for grades 1, 2, and 3: 
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Academic 

milestone 

Grade 

Potential 

Monitoring 

Indicator 

Threshold 

Research 

Supported 

Locally 

Determined 

Consider 

Using? 

Notes - existing data source, 
possible threshold, etc. 

Late Elementary 



Proficient or higher on grade 6 

ELA and mathematics State Assessment 

4 

Attendance 

20% or more school days missed 

X 




Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 




5 

Attendance 

20% or more school days missed 

X 




Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Course 

performance 

Failing grade mathematics and/or ELA 

X 




Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 




6 

Attendance 

20% or more school days missed 

X 




Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Course 

performance 

Failing grade in mathematics and/or ELA 

X 




Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 





Additional notes about possible monitoring indicators for grades 4, 5, and 6: 
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Academic 

milestone 

Grade 

Potential 

Monitoring 

Indicator 

Threshold 

Research 

Supported 

Locally 

Determined 

Consider 

Using? 

Notes - existing data source, 
possible threshold, etc. 

Middle School 



Passing grades in 
all grade 9 courses 

7 

Attendance 

20% or more school days missed 

X 




Course 

performance 

Failing grade in mathematics and/or ELA 

X 




Behavior 

Suspensions, office discipline referrals 


X 



8 

Attendance 

20% or more school days 

X 




Course 

performance 

Failing grade in mathematics and/or ELA 

X 




Behavior 

Suspensions, office discipline referrals 


X 



9 

Attendance 

10% or more school days missed 

X 




Course 

performance 

Failing one or more grading period 

X 




Credits 

Enough credits for promotion to next 
grade 

X 




Behavior 

Suspensions, office discipline referrals 


X 




Additional notes about possible monitoring indicators for grades 7 , 8, and 9: 
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Academic 

milestone 

Grade 

Potential 

Monitoring 

Indicator 

Threshold 

Research 

Supported 

Locally 

Validated 

Consider 

Using? 

Notes - existing data 
source, possible 
threshold, etc. 

High School 




10 

Attendance 

10% or more school days missed 

X 






Course 

performance 

Failing one or more grading period 

X 






Behavior 

Suspensions, office discipline referrals 


X 



o 

'■+-> 

D 

"O 

11 

Attendance 

Locally defined percentage of school 
days missed 


X 



03 

i_ 

CUD 

~o 

O 


Course 

performance 

Locally defined number of course 
failures 


X 



u 

CO 


Behavior 

Suspensions, office discipline referrals 


X 



WD 

ir 

12 

Attendance 

Locally defined percentage of school 
days missed 


X 





Course 

performance 

Locally defined number of course 
failures 


X 





Behavior 

Suspensions, office discipline referrals 


X 




Additional notes about possible monitoring indicators for grades 10, 11, and 12: 
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Tool 4 - Step 1: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 1. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 

A. Team Structure, Roles, and Responsibilities 

Key Tasks: 

• Determine whether team will be newly formed, or integrated into an existing committee or team. 

• Determine team structure (school level, district level, or combined school and district level membership). 

• Determine team composition (who will be part of the team). 

• Determine team roles and responsibilities (e.g., team facilitator, data management designee, interventions specialist, team 
logistics). 

• Determine meeting schedule (timing, frequency, and length). 

• Clarify team's group process (decision making, conflict resolution, and ground rules). 

• Communicate with district leadership to clarify goals and expectations (district and school). 

• Set team goals, objectives, tasks and timelines. 

• Construct a team communication plan. 


Objective or Task 

What do we have in place? 
What expertise and/or 
resources do we already 
have? 

What additional actions 

and resources are needed? 

Who will be responsible? 

What is the time frame? 
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B. Get Ready to Use Early Warning Data 

Key Tasks - EWIS Data and Reports: 

• Designate a person to manage EWIS data and generate 
EWIS reports. 

• Obtain permission to access EWIS data 

• Ensure that district and/or school student data are up- 
to-date. 

• Determine how student confidentiality will be 
protected. 

• Ensure that team members understand EWIS (risk 
levels, academic milestones) and how to interpret 
data. 


Key Tasks - Monitoring Indicators: 

• Designate a person to manage monitoring indicator data 
and generate reports. 

• Determine final list of monitoring indicators. 

• Establish thresholds for locally determined monitoring 
indicators. 

• Determine how student confidentiality will be protected. 

• Set schedule for updating monitoring indicator data and 
generating reports. 

• Ensure that team members understand how to interpret 
monitoring indicator data and reports. 


Objective or Task 

What do we have in place? 
What expertise and/or 
resources do we already 
have? 

What additional actions 

and resources are needed? 

Who will be responsible? 

What is the time frame? 
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Step 2: Review EWIS Data 


What You Need for Step 2 

■ Access to Edwin Analytics for generating EWIS reports 

■ Exploring your Early Warning Data Worksheet (see Tool 7 in Step 2) 

■ Sufficient time for team members to analyze existing data and reports and generate additional 
reports 

Description of Step 2 


In Step 2, the team explores EWIS data to answer important 
questions about students in need and understand patterns in 
student engagement and academic performance within a 
school and/or district. The data reports in Edwin Analytics are 
tools for seeking answers to the team's questions. 

When teams first start looking at the EWIS data, it can be 
overwhelming. It takes time to develop a strategy to review 
the information. During the first year of implementation, this 
can take a substantial amount of time; as the team becomes 
familiar with the process and the data, this will become more 
efficient. If EWIS data for students entering school are 
available, Step 2 may be started before the beginning of the 
school year. The process of reviewing data through questions 
outlined in Step 2 is critical when using any type of early 
warning early data. For example, when it is time to review 
monitoring indicators data (Step 5), it is recommended that 
your team use this same approach to reviewing data. The 



Build Off Your Foundation 


■ What questions or challenges at 
the school level or district level 
are you already focused on? 
Consider exploring these 
questions in your EWIS data 
review. 

■ Are there particular student 
groups your school or district has 
already identified as an area of 
challenge and could be the focus 
of the EWIS data review? 



following represents some guidelines for how teams may want to approach the data analysis process. 


Anticipated Outcomes for Step 2 


The following outcomes are anticipated for Step 2: 

■ An understanding of the nature and the scope of district and schoolwide student EWIS risk levels 
(i.e., the percentages of students at low, moderate, and high risk) 

■ An understanding of patterns associated with risk levels for students across the district and within 
and across schools (e.g., how risk levels vary by grade, gender, disability status, race or ethnicity, 
and other characteristics) 

■ Identification of individual and/or groups of students who show signs of risk of missing desired 
academic milestones 
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■ An understanding of patterns across a district, schools, and groups of students that allow the team 
to consider how to best allocate and direct interventions and supports to best meet student needs 
at the individual, group and schoolwide levels 


■ Initial hypotheses related to potential underlying causes of student need 

■ Tentative decisions about the scope and the focus for exploring underlying risk and identifying the 
types of additional information that the team may want to collect 


Exploring Data Through Questions 


Teams will want to begin by generating EWIS reports. It is important to recognize that EWIS (and, later 
on, monitoring indicators) reports can quickly yield a lot of information. As a result, it is very easy for the 
team to become overloaded with data and subsequently not be able to make sense of how to use the 
data or even what the data are telling them. As shown in Exhibit 6, teams can make the data exploration 
process more manageable by using three levels of examination: 

■ Get the big picture. Examine data by broad dimensions and main categories. 

■ Dig deeper. Examine data by subgroups or subcategories. 

■ Deep dive. Examine data for individual students. 


This multilevel examination can provide a context for understanding student needs within a district or a 
school and can help the team break down the information into manageable pieces that can be sorted, 


organized, and prioritized so that the team can take action. (See Tools 7 and 8 below for worksheets to 


guide your team's data review.) 


Notes from the Field 


We learned that it is best to start 
with the data and the questions 
we have rather than starting by 
saying we are implementing 
“EWIS. ” This way it doesn’t feel 
like something new — or one 
more thing to do. 
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Get Ready. Before getting started, the team develops a plan for how they will review their EWIS data 
(e.g., which questions to answer, how to focus analyses [whole school, subgroups, and individual 
students], and who will be responsible for reviewing which reports). For most districts and schools, it is 
not feasible to fully analyze all of the EWIS data; therefore, it is helpful to have specific goals and 
questions in mind to more clearly define the task. 

Get the Big Picture: Examine Data by Broad Dimensions. The team may want to begin reviewing its 
EWIS data by generating data reports that offer the big picture of student risk within a district or a 
school, such as the EWIS District View report (EW301). 

It is important to acknowledge that in some schools, a high percentage of students may be identified as 
being at risk. In these schools, the team must decide how best to proceed in understanding and 
addressing student needs and how to allocate time with respect to understanding and addressing the 
needs of specific groups of students and individual students. 

Examples of broad dimensions (main, single categories) for reviewing EWIS student risk levels by: 

■ School 

■ Grade level 

■ Academic milestone 
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Some big picture questions are as follows: 


■ What proportion of the district's or school's students are at low risk? Moderate risk? High risk? 

■ Are there differences across schools serving similar 
grade levels? 

■ What student groups may warrant further 
exploration? 

Before moving on to the next level of analysis (dig 
deeper), the team summarizes what has been learned 
so far and what questions still need to be answered in 
the next phase of analysis. 

Dig Deeper: Examine Data by Subgroups or 
Subcategories. After the team has a sense of the broad 
landscape (overall nature and scope of need), it is time to 
dig deeper into the data to uncover patterns of risk by 
each subgroup of students (e.g., gender, race or 
ethnicity, ELL status, special education status, grade 
level), and patterns of students by subcategories of risk 
(EWIS student risk levels, academic milestone, or by data used to determine EWIS student risk levels). This 
deeper analysis of EWIS data helps the team identify patterns associated with EWIS risk levels for 


Implementation Tip : 

Start with EWIS District View Report to 
get a Big Picture of Your Data 

Edwin Analytics EWIS District View Report 
(EW301) is particularly helpful for illustrating 
the big picture of student risk for your school or 
district. By broadly examining EWIS data, the 
team is able to assess the nature and the scope 
of student risk levels within the overall student 
population of interest, before delving more 
deeply into the needs of specific cohorts of 
students or individual students. Using the 
district view report offers the team a context in 
which to understand the extent of student risk 
within the district or school, as well as inform 
decisions about the future allocation of time 
and resources. 


particular subgroups of students across the district and across and within schools (e.g., how EWIS risk 
levels vary by grade, student subgroups). The team may also want to explore patterns associated with 



Notes from the Field 


We worked as a team to look at 
the data. First, our principal 
pulled the data and did a cursory 
analysis of trends. Next our team 
decided which subgroup to 
target, and we pulled info on 
those students so we could look 
at individual students. Next we 
asked how many high risk 
students are already assigned 
an intervention? For those 
students who weren't already 
receiving any supports, we 
assigned a staff person to follow 
up to find out more. 



particular data points that were used to determine the EWIS 
risk levels (e.g. How many students flagged as EWIS high risk 
failed mathematics last year? What is the attendance pattern 
for our high and moderate risk students? ). During the Dig 
Deeper phase, the team begins to identify sub-populations of 
students of concern or gathers more information about a 
student group already identified. 
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Examples of subgroups or subcategories for examining EWIS data include: 

■ Patterns of EWIS risk levels across and/or 
within student groups: 

• Student demographic characteristics 

• Student language and learning needs 

• Within each grade, age group category (e.g. late 
elementary) 

■ Patterns across data points used to determine EWIS 
student risk levels: 

• Attendance patterns 

• Suspension patterns 

• State assessment results 

• Course performance patterns 

Examples of subgroup or subcategory questions include: 

■ District level. How do student EWIS student risk 
levels differ across schools? For example, is the 
breakdown of low, moderate, and high risk 
consistent across grade 3 in our elementary 
schools? Are there EWIS student risk level differences among student subgroups, attendance 
patterns, etc. across the schools in the district? 

■ School level. What are the characteristics of students who have been identified as being at risk? 
What percentage of these students are ELLs, special education, and/or low income? What are the 
suspension patterns of students identified as high risk across the grade levels at our school? 

Important Note: For some districts and schools, a high percentage of students may be identified as 
being at high or moderate risk. In these cases, after districts and schools understand patterns associated 
with risk, they may choose to focus most of their efforts on addressing student needs on a schoolwide 
basis or providing supports for targeted subgroups of students. 


Implementation Tip : 

Take Advantage of the EWIS Report 
Options in Edwin Analytics 

Three of the EWIS reports in Edwin Analytics 
may be particularly helpful for the "dig deeper " 
level of examination: 

■ Subgroup Analysis (EW318) shows the 
breakdown of EWIS student risk levels by 
student subgroups, as well as a state 
comparison for the student subgroup 
populations 

■ Risk Level Indicator Analysis (EW317) 
provides attendance, behavior, state 
assessment, and course patterns for 
students in each EWIS student risk level for 
a selected grade level 

■ Graphical View (EW302) includes two line 
charts that provide two different 
longitudinal views for the selected 
students 


Before moving on to the next level of analysis (deep dive), the team summarizes what has been learned 
so far and what questions still need to be answered in the next phase of analysis. 
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Deep Dive: Examine Data for Individual Students. By now, 
the team should have a clearer understanding of which 
groups of students are in need. The deep dive requires that 
the team look beyond EWIS student risk levels (low, 
moderate, or high risk) to begin to examine the potential 
factors that may be contributing to the student's risk for 
not meeting key academic milestones. 

Teams can begin by using Edwin Analytics to examine the 
specific data points (e.g. attendance, course performance, 
suspensions) that contributed to a student being flagged as 
moderate or high risk. For example, the EWIS student view 
report (EW 601), provides all of the data points that were 
used to provide an EWIS student risk level for a given 

k, the team 
may find that 
the student 
failed math the 
previous year 

and attended school less than 90 percent of the school 
year. While these data points do not necessarily explain 
why a student is at risk, uncovering this additional 
information can help the team begin to formulate 
hypotheses that will guide the gathering of additional data 
to improve the team's diagnosis of the reason a student is 
in need and thereby make decisions about interventions 
and supports. 

Examples of school-level deep-dive questions: 

■ Who are the individual students identified as being at high 
risk from our EWIS data? 

■ What else can we learn about these individual students 
by looking at their individual data points in Edwin Analytics 
(report EW601)? 

■ Do we see any patterns in the data that can guide us in gathering more information? 


student. For one student considered high ris 


Notes from the Field 

Our team took EWIS and sorted 
into four different data files (by 
grade) and proceeded to color- 
code each grade level based 
upon students who had: already 
dropped out, had SPED 
services, had outside services, 
or who are in alternate 
programs. Anyone not color- 
coded and in the high risk group, 
were then the kids who had no 
one monitoring them. These 
were the kids who could possibly 
fall through the cracks. 



Notes from the Field 


We used EWIS data to gain the 
big picture on freshman 
repeaters. We dug deeper to 
identify that there were ELA 
course scheduling issues for 
many students. In addition, we 
did a deep dive to talk about the 
needs of individual students by 
providing guidance counselors 
with data on their assigned 
students. 




Preparing for Step 3 


After the team has a sense of the overall nature and the scope of student risk based on their review of 
the EWIS data, it is time to begin considering the reasons why students may be at risk and determine 
whether and what types of additional information may be needed to gain a more complete 
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understanding of the factors that may be contributing to this risk. The team should decide if that nature of 
student need warrants exploring full district and/or school changes, interventions and supports for 
subgroups of students, and/or additional study for individual students. 

It is suggested at this point that the team begin to generate initial hypotheses to explain underlying 
causes of risk and needs and articulate follow-up questions in each area of greatest concern. This 
process can be done for each of the three levels (whole school or district, subgroups of students, and 
individual students), or the team may decide to focus on only a few areas or with only a targeted group 
of students. For those districts and schools using early warning data for the first time, it is advisable to 
start small. 

Before moving on to Step 3, it is recommended that the 
team identifies its priorities, and make tentative 
decisions about the scope and the focus for exploring 
underlying causes. It also is helpful to develop a strategy 
for how the team will divide up responsibilities for 
gathering, reviewing, and interpreting additional data as 
they implement Step 4. 

District Role in Step 2 

The use of EWIS data has immediate implications for 
staff and students in a school; however, at the district 
level, there is an opportunity to examine EWIS data to 
illuminate broad trends in student risk that may 
influence how resources are allocated or policies and 
strategies are implemented to focus on issues that are 
particular to groups of students within a district. For 
example, what district-wide changes could be made to 
reduce the high numbers of incoming grade 9 students 
who are identified as high risk? 

Guiding Questions for Step 2 

■ What are the overall nature and the scope of districtwide and schoolwide EWIS student risk levels 
(i.e., the percentage of students at low, moderate, or high risk)? 

■ What patterns are associated with student risk levels for subgroups of students across the district 
and within and across schools (e.g., how risk levels vary by grade, gender, race or ethnicity, and 
other characteristics)? 

■ Which groups of students require further digging? What are their characteristics (e.g., grade, 
gender, and ELL status)? What data points (e.g., attendance and course performance) are most 


Implementation Tip : 

Start Small 

Districts and schools using early warning data 
for the first time may want to learn more about 
EWIS and early warning implementation before 
attempting to serve the needs of all students 
identified as being at high or moderate risk in 
the district or the school. 

There are several ways to start small. Some 
schools may want to begin by focusing on those 
students who will be transitioning to a new 
school, such as grade 6 students entering 
middle school. Others may want to address the 
needs of a particular cohort of students (e.g., 
ELLs) or concentrate on those students who 
were identified as being at high risk. Keeping 
the scope manageable will allow the team to 
spend time refining its process so that it is able 
to operate efficiently and effectively, serving 
larger numbers of students in later years. 
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concerning for these students? How have patterns of risk changed over time for the district, schools, 
subgroups, and individual students? 


■ Given our assessment of student risk, what additional questions and information will we need to 
gather at the district, school, subgroup, and individual student levels to more fully understand the 
potential underlying causes for students' risk and schoolwide needs? 
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Tool 5 - Step 2: Checklist 


Get Ready: 

□ Confirm scope and goals for reviewing EWIS data (e.g. are there particular district or school 
priorities you want to consider when reviewing your data) 

□ Develop a plan for how you will review EWIS data (e.g., which questions you will answer, how 
you will focus your analyses) 

□ Determine who will be responsible for generating reports, reviewing report 

Get the Big Picture: 

□ Generate District View Report (EW301) 

□ Review district or schoolwide data to determine schools, grade levels or sub groups of interest 
for further exploration 

Dig Deeper: 

□ Generate Graphical View (EW302), Risk Level Indicator Analysis (EW317) and Subgroup Analysis 
(EW318) reports 

□ Explore patterns related to student risk for sub groups of students 

□ Identify subgroups and individual students of concern 

Deep Dive: 

□ Generate Student List (EW601) reports 

□ Review data for individual students and small sub groups of students 

□ Identify specific students and data related to these students that the team would like to better 
understand 

Generate Hypotheses and Set Priorities: 

□ Identify any schoolwide risk patterns 

□ Identify subgroups of students at-risk, and identify any patterns associated with risk for these 
groups of students 

□ Confirm the list of students and student groups the team is concerned about, and articulate the 
team's theories about what may be contributing to risk for these students 

□ Establish the team's priorities for exploring underlying causes of risk 

□ Identify any additional questions and information the team will need to gather to more fully 
understand potential underlying causes for students' risk and schoolwide needs 
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Tool 6 - Step 2: Practice Generating Reports 


Directions: Generate one EWIS report for each of the five reporting options: District View (EW301), 
Graphical View (EW302), Risk Level Indicator Analysis (EW317), Subgroup Analysis (EW318), and Student 
List (EW601). 

1. District View (EW301): Select a school in your district and generate a report for all grades in this 
school. Scan your report to find out which grade has the highest percentage of "high risk" 
students and which grade has the lowest percentage of "high risk" students. 


2. Graphical View (EW302): Select a grade level and/or student group of interest. Generate a 
graphical view report. Find out how this group compares with the overall district and state. Find 
out how risk levels for the current group of students in this grade have changed over the past 
year. Find out how risk levels for this grade, generally, have changed. Are risk levels for this 
group of students getting better? Worse? Staying the same? 


3. Risk Level Indicator Analysis (EW317): Select a grade level of interest, and run a Risk Level 
Indicator Analysis report. What attendance, suspension, assessment results, or course 
performance patterns do you observe for this grade level? 


4. Subgroup Analysis (EW318): Select a grade level and/or student group of interest, and run a 
subgroup report to display how risk levels for this grade and/or student group vary across the 
subgroups. Which of these subgroups shows the highest level of risk? 


5. Student List (EW601): For your selected grade and/or subgroup of students, generate a Student 
View report to view some of the individual students who are at high risk. What additional 
information do you observe when viewing the individual students? 
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Tool 7 - Step 2: Exploring Early Warning data by Broad Dimensions/Main Categories 


A. Big Picture: Here are some sample "Big Picture" questions. You may choose questions from the list below to explore, or devise your own 
questions. 


Sample Broad Questions 

EWIS Data & Reports 

EWIS Data Output Percent or Number of Students 

1. What proportion of our student body is at 
low, moderate, high risk? 

District View Report (EW301) 

High 

Moderate 

Low 

2. How do our student risk levels vary by 
school level (elementary, middle, and high 
school), by school, or by grade across or 
within schools? 

District View Report (EW301) 

High 

Moderate 

Low 

3. How do our students' risk levels differ by 
risk level indicator categories? 

■ Attendance 

■ Suspensions 

■ MCAS results 

Course performance 

Risk Level Indicator Analysis 
(EW317) 

High 

Moderate 

Low 

4. How have student risk levels changed over 
the past year? Over the past two years? 

Graphical View Report (EW302) 

High 

Moderate 

Low 

5. How do risk levels differ according to 
student characteristics, across or within 
schools? 

■ Gender 

■ High-need students 

■ ELL students 

■ Students with disabilities 

■ Overage students 

■ Other 

Subgroup Analysis (EW 318) 

High 

Moderate 

Low 


Use the Data Exploration Summary Chart to reflect on what you have you learned. Which grades, student groups, or indicator 
categories would you like to explore further? 
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B. Dig Deeper: Exploring EWIS Data by subgroups/sub-categories 


Directions: Identify one student group, school, or grade, of concern to explore further (e.g., middle school boys or students flagged due 
to attendance.) Here are some sample “Dig Deeper" questions for exploring the data related to your target group of concern. You may 
choose from the questions listed below, or devise your own questions. Important Note: Repeat this step for each targeted group of 
concern. 


Sample Subgroup/Category Questions 

EWIS Data & Reports 

EWIS Data Output Notes Percent or Number of Students 

1. How do risk levels for (selected 

student group) vary by additional student 
characteristics (e.g., gender, income, ELL status, 
disability status.) 

Subgroup Analysis (EW318) 

Risk Level Indicator Analysis 
(EW317) 

High 

Moderate 

Low 

2. Within our student group of concern, how many 
individual students are identified as "high risk"? 

As "moderate risk"? 

District View (EW301) 

High 

Moderate 

Low 

3. How have risk levels for our student group of 
concern changed overtime? 

Graphical View (EW302) 

High 

Moderate 

Low 


Use the Data Exploration Summary Chart to reflect on what you have you learned from reviewing your EWIS data. Which particular 
students are you most concerned about? 
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C. Deep Dive: Exploring early warning data by individual students 


Directions: Identify individual students of concern to explore even further.* 


Individual Student-Level Questions 

EWIS Data & Reports 

EWIS Data Output Notes about Students 

1. Looking at the refined subgroup of students 

Risk Level Indicator 


identified, what can we learn about their shared 

Analysis (EW317) 


risk level indicators? 



■ Attendance 

Student View (EW601) 


■ Behavior 



■ Course performance 

Export data to Excel to 


■ State assessment test performance 

explore shared risk level 


■ Previous year's risk level 

indicators for groups of 
individual students you 
are concerned about. 


2. What can we learn about patterns in the 

Subgroup Analysis 


characteristics of these students? 

(EW318) 


■ Grade? 


■ Learning and language profile? 

Student View (EW601) 


■ Other demographics (e.g. race/ethnicity) 



■ Other student characteristics? 

Export data to Excel to 
explore shared grade/age 
or other learning or 
demographic 
characteristics for groups 
of individual students you 
are concerned about. 



Use Data Exploration Summary Chart to reflect on what you have you learned. 


* When preparing for Step 3, schools will also want to identify students by name and classroom to explore potential underlying causes for risk. 
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Tool 8 - Step 2: Early Warning Data Exploration Summary Chart for EWIS Data 


Directions: Use this chart to take notes as you examine your EWIS data. 



Key Findings 

Areas of Concern 

Big Picture: Main Categories 

■ Overall risk levels 

■ Comparison with state 

■ Comparison with district 



■ By grade 

■ By school 

■ By year 



District and/or schoolwide by student groups and 
characteristics: 

■ Gender 

■ Race/ethnicity 

■ Low income 

■ ELL 

■ FELL 

■ SPED 

■ High Needs subgroup 



District and/or schoolwide by risk level indicator 
category: 

■ Attendance 

■ Suspensions 

■ State Assessment performance 

■ Course performance 




Early Warning Implementation Guide 


Page 48 





Key Findings 

Areas of Concern 

Dig Deeper: Subgroups/Sub-Categories 

By student groups within and across grades: 

■ Gender 

■ Race/ethnicity 

■ Low income 

■ ELL 

■ FELL 

■ SPED 

■ High Needs subgroup 



By risk level indicator category within and across 
grades: 

■ Attendance 

■ Suspensions 

■ Course subject Pass/ Fail 

■ State Assessment test performance 



By student groups within/across other student 
groups and categories (e.g., ELL/SPED students with 
low MCAS): 

■ Gender 

■ Race/ethnicity 

■ Low income 

■ ELL 

■ FELL 

■ SPED 

■ High Needs subgroup 

■ Attendance 

■ Suspensions 

■ Course performance 

■ State Assessment performance 
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Deep Dive: Individual Students 

Shared risk indicators and/or shared characteristics 
of individual students of concern: 

o Gender 
o Race/ethnicity 
o Low income 

o ELL 

o FELL 

o SPED 
o High Needs 
o Attendance 
o Suspensions 
o Course subject Pass/ Fail 
o State Assessment performance 
o Other characteristics: 
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Tool 9 - Step 2: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 2. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 

Key Tasks: 

• Generate EWIS data reports. 

• Determine key questions and/or focus for data exploration. 

• Determine the data analysis plan (who will participate in exploring data, when, for which data?). 

• Examine data at multiple levels (Big Picture - district and/or school level; Dig Deeper - subgroups; Deep Dive - individual students). 

• Identify patterns and summarize key findings (e.g., complete the data summary chart). 

• Identify schoolwide/district-wide groups of students, and/or individual students of concern. 

• Articulate initial theories or hypotheses related to the findings. 

• Identify follow-up questions to be answered with existing and/or additional data. 


Objective or Task 

What do we have in place? 
What expertise and/or 
resources do we already 
have? 

What additional actions 

and resources are needed? 

Who will be responsible? 

What is the time frame? 
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Step 3: Explore Underlying Causes 
What You Need for Step 3 


■ EWIS data and reports 

■ When available: Monitoring data and reports showing student risk flags 

■ A list of follow-up questions raised by the EWIS data analysis conducted in Step 2 

■ Potential Underlying Causes Worksheet (See Tool 11). 

■ A strategy for how the team will divide up responsibilities for gathering, reviewing, and interpreting 
additional data 

■ Sufficient time for the team to meet and review additional student data and discuss findings 


Description of Step 3 

Step 3 is about uncovering the reason(s) why students are at 
risk for not meeting academic milestones. It is important to 
acknowledge that EWIS risk levels and monitoring flags are 
merely signs or symptoms of deeper and likely more complex 
problems related to student achievement. Thus Step 3 builds 
on the review of early warning data by encouraging the team 
to explore potential underlying causes for individual students, 
groups of students, the school, and the district. Step 3 relies 
on a combination of data and professional judgment in 
assessing student needs. 

Anticipated Outcomes for Step 3 

The following outcomes are anticipated for Step 3: 

■ Additional diagnostic data for individual students or groups of students identified in Step 2, which 
can be used to discern the most likely underlying causes for risk 

■ A better understanding and evidence to discern the underlying reasons that individual students or 
groups of students may be at risk of not meeting academic milestones 

■ Identification of individual and common needs among groups of students and schoolwide 

■ Initial ideas related to the nature or the types of interventions and supports students may need to 
help them get back on track 



Build Off Your Foundation 


• How do you currently determine 
the reasons why students are 
struggling? Who participates in 
this work? 

■ Which data sources do you use? 
What aspects of school context 
and/or student functioning do 
you generally consider when 
investigating student needs? 

■ How can you integrate efforts to 
determine the reasons for 
students' EWIS risk levels with 
existing efforts? 

STEP 3 

I I I I I I I 
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Moving From Data to Action 


As shown in Exhibit 7, Steps 2, 3, and 4 are about moving from data to action for students. In Step 2, the 
team reviewed EWIS data to identify patterns (A) and formulated initial ideas and follow-up questions 
(B). As shown in Exhibit 7, the team engages in two key tasks in Step 3: gathering and interpreting 
additional diagnostic data to identify likely factors contributing to underlying risk (C) and using these 
data as evidence to confirm likely causes of risk and the likely needs of students (D). In Step 4, the team 
takes action by making schoolwide changes, and assigning individual and group supports (E). 

Exhibit 7. Moving From Data to Action 




E. Identify and 
implement 
schoolwide 
changes, and 
interventions 
and supports 
for groups and 
individual 
students. 



C. Collect 
additional data. 


D. Identify key 
problem(s) to be 
addressed. 


•Use multiple 



B. Prepare 


•Identify new 
data sources, 
who will gather. 
•Revisit A, 
discuss patterns 
in data. 


sources of data 
to support this 
conclusion. 


hypothesis of 
key issue. 


• Was hypothesis 
confirmed? 


A. Review 
data to 
identify 
patterns. 


Targets: 

•Group? 


•Individuals? 
•Schoolwide? 
• Grade level? 


Step 2 



Step 3 



Step 4 
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Identifying Potential Factors Underlying 
Student Risk 

After the team has identified students in need (schoolwide, 
groups, or individual students), the team then explores the 
potential underlying reasons for student need by examining 
student strengths and challenges, the nature of their 
classroom or school environments, and, potentially, outside 
contextual factors. 

This exploration takes into account that student success is 
dependent on more than just academic knowledge and 
skills. The reasons why students struggle or thrive in school 
stem from a complex interplay of individual; 

social; 


emotional; 
instructional; 
organizational; 
and other 


Notes from the Field 


We discovered that the reasons 
for student risk are far more 
complex than we may have 
realized in the past. 


Notes from the Field 

How we went about interpreting 
our EWIS data was not 
necessarily any different than 
our approach for understanding 
student needs in the past. 
However, by using early warning 
data, the size of the pool of 
students in need has increased. 
So, the process has become 
both harder and easier: Harder 
because there’s so much more 
data; easier in that we now have 
more pieces of data all saying 
the same thing. 


contextual factors, such as families, peers, and 
communities. 7 Research on factors that contribute to 
resiliency, 8 skills needed for the 21st century, 9 and 
emerging research on the brain 10 further support the notion 
that learning and student performance are dependent on 
the complex interaction between multiple domains of 
development and functioning. These, in turn, interact with 
contextual factors. For instance, five students all identified 
through EWIS data as high risk and all with attendance 
issues may have five distinct reasons (individual and 
contextual) for displaying symptoms of need and may 
require five different interventions to meet these needs. 


As the team explores the potential underlying causes for student risk, the team may find it beneficial to 
consider the range of student needs and the abilities that contribute to school success. As shown in 
Exhibit 8, individual development can be conceptualized as encompassing four key domains: physical, 
intellectual or cognitive, psychological or emotional, and social. 11 Exhibit 8 also illustrates a sample 
framework for identifying students' specific strengths, abilities, attitudes, and functioning within each 
area that may be contributing to or inhibiting school performance. As the team explores the potential 
underlying causes of student need, a conceptual model such as this one may help the team understand 
the needs of students and become more strategic about efforts to collect additional data. 
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Exhibit 8. Domains of Student Development and Functioning 


Cognitive 


Academic content, 
current knowledge 
and skills 


i 


21st century skills, 
including executive 
functioning 




Attitudes, beliefs, 
and motivation 


vior 


Social 


Individual learning 
profile, culture, 
language, abilities, 
and interests 


Emotional 
functioning and 
mental health 


Social competence 
and relationships 
with adults/peers 


Emotional or 
Psychological 


Students do not function in isolation. Rather, research 
suggests that student performance is the result of each 
individual's interaction with his or her environment . 12 
Effective learning environments are composed of a 
complex combination of factors . 13 For instance, research 
suggests that learning is more likely to take place in a 
supportive social environment . 14 Research also suggests 
that learning is enhanced with appropriate structure (i.e., 
when clear rules, routines, and expectations are balanced 
with opportunities for choice and autonomy). Students 
also learn best in an environment that offers ample 
opportunities for challenge and skill building, which will 
help students stay engaged and focused on their 
learning . 15 Teams may want to consider all of these factors 
as they explore underlying causes of student risk. Given 
the complexity of potential underlying individual and 
contextual causes, the team may then want to focus on the 
areas in which a school or its teachers have the greatest 
opportunity to make a positive change in student 



Notes from the Field 


We were able to add a qualitative 
flavor to our data interpretation by 
including survey data from 
teachers for the 30 at-risk kids we 
had identified. Using this 
additional data we could now talk 
about individual student 
performance and we could drill 
down deeper to better understand 
why students were at-risk by 
looking at information such as 
their attendance, whether they 
were doing their homework, and if 
they were completing long-term 
projects. 
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performance. (See Tool 11, Potential Underlying Causes reference sheet.) 


Gathering Additional Information 

To fully understand underlying factors in student risk, typically, the team needs to gather more data. 
However, before gathering further data, each team will want to be clear about its priorities, focus, and 
available time. To know where to start when collecting data, the team may want to begin by generating 
hypotheses for potential underlying reasons for risk related to its priority areas of concern— whether 
schoolwide, for groups of students, for individual students, or all three. To guide data collection, these 
hypotheses can be translated into questions the team would like to answer. For instance, a school that 
discovers a disproportionate number of grade 3 students at high risk may hypothesize that more needs 
to be done in grades K-2 to enhance students' literacy skills. So this team might ask the following 
questions: 

■ What does our K-2 literacy benchmark data tell us about student performance in key literacy skill 
areas? 

■ Which students in grades K-2 need additional literacy supports— and in what areas? 

■ How are we currently supporting student literacy in K-2? 

■ Is our reading curriculum and appropriate for our student population? 


Next, the team needs to determine the methods and the 
sources of information the team will use to collect data to 
answer its questions. To manage the amount of time 
required, the team identifies those responsible for gathering 
this additional information. In some cases, this may mean 
asking others outside the team to assist. The team also may 
want to establish a timeline for bringing the group together 
to examine findings, along with additional sources of data in 
preparation for Step 4. 

Schoolwide Data Collection. During Step 2, the team 
identifies a focus, priorities, and the scope of further study. 
For instance, a school with large numbers of students in need 
may have decided that data collection will focus on 
understanding factors that influence the whole-school 
environment. This school may want to conduct a learning 
walk-through, where educators select an area of inquiry and 
systematically collect data to answer their schoolwide questions 
(see http://www.doe.mass.edu/apa/dart/walk ) or conduct a focus group with school counselors. 
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Subgroup or Cohort Data Collection. Another school might discover in Step 2 that many of its students 
who are at high risk failed mathematics the previous school 
year, and monitoring flags also point to concerns with 
mathematics in the current school year. As a result, the 
team may gather additional information about the current 
mathematics curriculum, methods of instruction, teacher 
quality, and the supplemental supports that are available. 

Individual Student Data Collection. For schools with fewer 
students in need, or where students are not responding 
positively to interventions and supports, the team may want 
to obtain additional information to help diagnose the specific 
needs of individual students. Many teams begin by gathering 
information about students directly from those adults who interact with these students on a regular 
basis (e.g., classroom teachers, school counselors, and specialists). For example, for students who are 
identified as they enter grade 1 as being at high risk for not being able to read by the end of grade 3 
(scoring proficient or higher on the grade 3 ELA State Assessment), the team may need to gather 
information on students' kindergarten performance from the prior year's teacher. Often, a team also 
accesses existing information that is available through school data systems and locate other 
performance or assessment data for the student. 

Finally, in some cases, the team may determine that it is appropriate to conduct (or work with classroom 
teachers to conduct) one-on-one meetings with individual students, their family members, or both as 
appropriate. In-person data-gathering efforts with educators, students, and families shed light on the 
reasons students are identified as being at high or moderate risk or are flagged for a monitoring 
indicator and also can spark interest and opportunities to engage students and the adults who interact 
with them in providing additional interventions and supports. If the school is implementing Individual 
Learning Plans (ILPs) with students, the ILP process is an opportunity to have a meaningful discussion 
with the student (and possibly also the family) to learn more about the student's needs, strengths, and 
barriers. 

Most importantly, the additional information gathered during Step 3 improves the team's understanding 
about why students are identified as being at risk. The guiding questions for Step 3 suggest a sequence 
of inquiry steps. 

Interpreting Data 

It is easy to confuse a symptom with the cause for that symptom. For instance, when a child exhibits 
disruptive behavior in the classroom, a teacher is likely to implement a behavior management plan. 
However, simply addressing the behavior itself is unlikely to be effective in the long run if the reasons 
for a child's behavioral struggles are not fully understood. Perhaps the child is bored, is overwhelmed, 
has a disability, is having difficulty understanding the material, or is distracted by teasing from peers. 


Implementation Tip : 

How Much Data Are Enough? 

It is easy to keep identifying additional 
questions and data sources that the team 
would like to gather. To manage this 
process and avoid amassing too much 
data, aim for gathering three sources of 
information for each student or group of 
students and be sure the team is clear 
about what questions it hopes each source 
of data will answer. 
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Until educators can systematically and thoroughly interpret 
the early warning data they have gathered so that they can 
better understand the underlying causes for student 
difficulties, appropriate interventions and supports cannot 
be devised. In addition, although individual students 
respond to stressors in unique ways, it is likely that the 
underlying cause(s) for one student is linked to broader 
areas in need of improvement in a classroom, a school, or a 
district that, if addressed, would benefit many students. 

After gathering additional information, the team needs to 
interpret what the data mean. To begin, the team reviews 
its original EWIS reports, (and when available monitoring 
flags) for schools, groups of students, and/or individual 
students. Teams combine these data with additional 
diagnostic data that has been gathered to discern potential 
underlying causes of student need. As the team reviews 
data and conducts the analysis, it is suggested that the - MM-iyUrw ^ 

team openly discuss any previously held assumptions about 

individuals or groups of students and replace those assumptions with ideas about likely underlying 
causes that are based on factual evidence. 

Confirming Student Needs by Using Evidence 

During Step 3, it is likely that the team will develop new ideas for the underlying cause(s) for student 
risk. On the basis of these investigations, the team should be able to identify some common and 
individual needs among students and prepare to identify and implement appropriate intervention 
strategies (Step 4). Therefore, the last three tasks in Step 3 are to: (1) clarify and confirm that the team's 
hypothesis is based on the collected evidence; (2) identify specific support goals for students, groups of 
students, or the whole school; and (3) establish initial priorities for how student and schoolwide needs 
will be met. It is critical to designate enough time for these tasks. Some teams may want to designate 
subgroups or work with other educators and/or classroom teachers to address particular students or 
groups of students and then reconvene as a full group to make final decisions and plan action. Before 
moving on to Step 4 it is important that the team establishes priorities for meeting student needs so 
decisions about the use of available resources needed for interventions can be made most efficiently. 

Important Note: The first time the team meets to discuss underlying causes of student risk will require a 
substantial amount of time. As the team clearly identifies sources for student information and questions 
to consider for understanding student needs, it will become more skilled at the process, so the time 
needed to conduct Step 3 will decrease. 



Notes from the Field 


Who are our students who are 
really struggling? Historically, we 
had always focused on ELL and 
students with disabilities. 

Looking at our EWIS data we 
saw that many of our at-risk 
students weren’t students with 
disabilities and were non-ELL. 
These were students who were 
struggling but not receiving 
supports. 
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District Role in Step 3 


Districts have an important role to play in supporting a school's additional data collection efforts. Often, 
schools need to access information beyond the EWIS information housed in Edwin Analytics and/or the 
monitoring indicator system. For instance, a school may want data from its feeder schools. District 
administrators can support these efforts by meeting with teams to understand and support their local 
data collection and developing policies that give the team members or designees access to information 
so that they can efficiently and independently gather information. In some cases, the district may want 
to participate in data collection efforts or conduct its own (additional) data collection efforts. A district 
also can support its school-level teams by providing sufficient time and staff coverage to allow the 
school-level teams to engage in the time-intensive process of exploring the underlying causes of risk for 
students. 

Guiding Questions for Step 3 

■ Overall guiding questions 

■ About which student(s) are we most concerned? What do we already know about them? 

■ What are the team's initial theories or hypotheses about the most likely underlying causes of 
student need? 

■ Are there schoolwide concerns related to patterns in student risk? 

■ What are the team's next steps for collecting additional information? 

• What types of information will be most helpful in better understanding the most important 
underlying causes for the risk status of students? 

• What sources (e.g., learning walkthroughs, teachers, school counselors, assessment data, 
student interviews) will the team gather information from? 

• Who will collect the information and when? 

■ Based on the evidence collected, what does the team believe are the most important underlying 
causes of student need? 

• For individual students 

• For groups of students 

• At the school level 

• At the district level 

■ What are initial ideas for potential interventions and supports for the whole school, groups of 
students, and individual students? How will you prioritize these needs? 
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Tool 10 - Step 3: Checklist 


Define the Target: 

□ Confirm which at-risk student(s) the team is most concerned about 

□ Clarify and document what the team already knows about these students 

□ Articulate hypotheses related to causes of risk for these students 

Gather Additional Data: 

□ Identify what additional information needs to be collected to better understand underlying 
causes of risk for identified students 

□ Determine which data sources will yield the best information to set data collection priorities 

□ Determine how and who will collect these data (i.e. create a Data Collection Plan ) 

□ Review and analyze the additional data 

Confirm Likely Underlying Causes and Student Needs: 

□ Determine what has been learned about underlying causes of risk from new data or evidence; 
confirm likely cause(s) 

□ Determine what at-risk student(s) need (define the problem to be solved) 

□ Generate initial ideas for appropriate interventions and supports 
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Tool 11 - Step 3: Potential Underlying Causes of Risk and Associated Data 
Sources Reference 

Directions: Use this optional reference sheet to help identify the potential underlying causes of risk for a 
student or group of students of concern. Causes of risk may due primarily to individual student, 
contextual or other factors or a combination of factors. Once you have identified potential causes of 
risk, select the data sources you will use to further explore the likelihood that these factors are 
contributing to risk for a student or students. 


Potential Underlying Factors: 

Student-Level 

Potential Data Sources 

Existing Data Sources We Could 
Use 

New Data We 

Might Collect 

Are academic or cognitive factors 
contributing to risk? 

O Academic skills or content knowledge 
(by subject) 

O Transferable learning skills (e.g., 

organization, problem solving, decision 
making) 

O Learning profile (preferred learning 
modes or methods, information 
processing) 

O Language proficiency 

O Special learning needs (academic, 
cognitive, or executive functioning) 

O Other 

O EWIS student risk levels 

O Student data in other Edwin 

reports 

O Monitoring Indicator Flags 

O Student attendance 

O Grades (current and prior) 

O Teacher progress reports 

O Grade retention 

O State Assessment scores 

O Benchmark assessment data 

O Subject area tests 

O Language proficiency tests 

O Special education testing 
results 

O Other 

O Teacher conversations or 

interviews 

O Student interviews or surveys 
O Family interviews or surveys 

O Additional academic 

assessments 

O Afterschool or community 
program staff interviews or 
surveys 

O Other 

Are emotional or psychological factors 
contributing to risk? 

O Attitudes and beliefs about self and 
own competencies 

O Intrinsic motivation 

O Growth versus fixed mindset 

O Mental health issues (anxiety, 

diagnosed mental health condition or 
issue) 

O Witness to or victim of a traumatic 

event 

O Special needs (emotional, 
psychological, behavioral) 

Are social competence and relationship 
factors contributing to risk? 

O Lack of social skills or competencies 

O Difficulty understanding social norms 
for school and classroom 

O Difficulty relating to peers 

O Difficulty relating to teachers and other 
adults 

O Teacher progress reports 

O Special education testing 
results 

O Results of other assessments 

O Suspensions 

O Behavioral records 

O Student attendance 

O School counselor records 

O Medical reports 

O School nurse reports 

O Doctor reports or letters 

O Letters or notes from family 

O Other 

O School counselor, social 
worker, or psychologist 
interviews or reports 

O Teacher conversations or 

interviews 

O Classroom or other 

observation 

O Student interviews or surveys 
O Family interviews or surveys 

O Afterschool or community 
program staff interviews or 
surveys 

O Additional assessments 

O Other 
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Potential Underlying Factors: 

Student-Level 

Potential Data Sources 

Existing Data Sources We Could 
Use 

New Data We 

Might Collect 

Are physical health or well-being factors 
contributing to risk? 

O Chronic or acute health issues 

O Fatigue or insufficient sleep 

O Difficulty managing stress 

O Risky behaviors (drugs, alcohol) 

O Physical handicap or challenge 
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Potential Underlying Contextual Factors: 

School or Classroom Level 

Potential Data Sources 

Existing Data Sources 

We Could Use 

New Data We 

Might Collect 

Classroom level 

o 

Teacher or classroom 

o 

School or classroom 

o 

Subject area 


assignments 


Learning Walkthrough 

o 

Level of rigor of class or classes 

o 

Academic aide assignments 

o 

Principal interviews or 

o 

Curriculum/sequence for course 

o 

Class size and composition 


reports 

o 

Teacher match to students' individual 

o 

Student-teacher ratios by 

o 

Teacher conversations, 


learning needs (e.g. personality. 


classroom or grade 


focus groups, interviews, or 


instructional approach, modality, level 

o 

Subject area level (e.g., 


surveys 


of challenge too difficult or too easy. 


remedial, standard, 

o 

Student focus groups, 


pacing) 


accelerated) 


interviews, or surveys 

o 

Relevance or match to student 

o 

Curriculum and lesson plans 

o 

Family focus groups, 


interests and goals. 


for subject or class 


interviews, or surveys 

o 

Student-teacher ratio 

o 

School or class schedule 

o 

Other 

o 

Size or composition of class(es) 

o 

Teacher website 



o 

Classroom match to student learning, 

o 

Homework assignments 




social and emotional needs 

o 

School policies and 



o 

Time of day of class 


procedures 



o 

Volume or type of homework 

o 

Teacher attendance 



o 

Peer mismatches 

o 

Classroom attendance or 



o 

Other 


participation records 





o 

Other 



School level 





o 

Curriculum 





o 

School Schedule 





o 

Pace or transitions 





o 

Policies and procedures for matching 






students with teachers 





o 

Discipline policies 





o 

Protocols for managing students 






moving in, out, and between schools 





o 

Other 
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Potential Underlying Factors: 

Other Factors 

Potential Data Sources 

Existing Data Sources We Could 
Use 

New Data We Might Collect 

Pragmatic or logistical issues: 

O Transportation to and from school 

O Class scheduling conflicts 

O Lack of appropriate place or time for 
homework 

O Lack of access to internet 

O Outside work responsibilities such as 
caring for siblings, employment 

Challenges outside of school: 

O Mobility 

O Homeless 

O Issues with peers/gangs 

O Neighborhood crime or safety 

O Family stressors, issues 

O Other 

O Student attendance 

O School or class schedules 

O Bus schedule or pick up and 
drop off locations 

O Teacher website 

O Homework assignments 

O Mobility records 

O Other assessments (e.g., 

Youth Risk and Behavior 
survey) 

O Crime or police records or 
reports 

O Other 

O Principal interviews or 
reports 

O Other school administrator 
interview or report 

O Teacher conversations, 
focus groups, interviews, or 
surveys 

O Student focus groups, 
interviews, or surveys 

O Family focus groups, 
interviews, or surveys 

O Afterschool or community 
program staff interviews or 
surveys 

O Other assessments 

O Other 
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Tool 12 - Step 3: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 3. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 

Key Tasks: 

• Confirm students and/or target group(s) of concern. 

• Clarify hypotheses and identify any questions to be answered with additional data. 

• Identify suitable and sufficient data sources for each area, student, or group of concern. 

• Determine who will collect these data and when. (How the team will divide up responsibilities for gathering, reviewing, and interpreting 
additional data?) 

• Review existing and additional data to discern likely underlying cause(s) (i.e., root cause analysis). 

• Confirm the problem to be addressed and cite evidence to support your conclusion. 

• Identify student needs and support goals that the team plans to address. 

• Identify initial ideas for the types of interventions and supports that will be needed. 


Objective or Task 

What do we have in place? 
What expertise and/or 
resources do we already 
have? 

What additional actions 

and resources are needed? 

Who will be responsible? 

What is the time frame? 
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Step 4: Take Action 
What You Need for Step 4 

■ An understanding of student needs in the school, based on evidence gathered by reviewing EWIS 
data and monitoring flags as well as additional information gathered in Step 3 

■ Identified support goals for students, groups of students, and/or schools 

■ Initial priorities for how student needs will be met 

■ If applicable, information regarding your district/school's procedures for implementing a tiered 
system of support 

■ Tools 14 and 15, Interventions and Supports Inventory and Review 

■ Appendix D: Targeted and Schoolwide Interventions Resources 


Description of Step 4 

Step 4 is about taking action. In this step, using the data 
from Step 3, the team identifies appropriate interventions 
and supports to align with the needs of students on an 
individual, group, and/or schoolwide basis. During Step 4, 
the team creates an inventory of available academic and 
non-academic interventions and supports in the school, the 
district, and the community and identifies and addresses any 
gaps in those supports. Next, the team considers how best to 
allocate and assign resources to meet student needs, which 
can be approached through a tiered continuum of academic 
and nonacademic supports. As the team identifies 
appropriate interventions and supports, the team may draw 
on available resources, adopt and offer new programs, 
and/or implement other schoolwide changes. Finally, the 
team works with district and/or school educators and 
students to ensure that there is sufficient intensity, 
oversight, and commitment for the interventions and 



Build Off Your Foundation 

How do you currently identify 
interventions and supports for 
students? 

Do you use a tiered system of 
support? 

Which current teams are 
involved in identifying and 
providing interventions and 
supports? In making schoolwide 
improvements? 

How can you integrate the way 
you provide supports and 
interventions to students and 
address schoolwide 
improvements? 


Ill 


i i 


STEP 4 




supports to be successful. 


Anticipated Outcomes for Step 4 


The following outcomes are anticipated for Step 4: 

■ A compiled inventory of academic and nonacademic interventions and supports available to 
students within and outside of the school 
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■ The identification of gaps in the available interventions and supports for students and 
recommendations for new strategies 


■ Information regarding potential evidence-based interventions and supports for identified student 
needs (available or desired) 

■ The provision of interventions and supports on the basis of the student needs identified in Steps 2 
and 3 (documented for each individual student). Districts/schools may integrate this with their 
approach to assigning universal, Tier II, and III supports and interventions through their tiered 
system of support 

■ A plan for tracking and assessing the success of interventions and the fidelity of implementation 

■ A plan for monitoring student progress and adjusting interventions and supports 

■ Recommendations for grade, schoolwide, and districtwide core or universal changes and support 
strategies aimed at addressing the most common student needs identified in Steps 2 and 3. 
Districts/schools may integrate this with their approach to Tier I interventions through their tiered 
system of support 

Using a Systematic Approach to Assigning Interventions and Supports 

In many schools and districts, interventions and supports for students are available but could be more 
systematically delivered and better coordinated. The use of EWIS data and monitoring indicators can 
be the foundation for developing or furthering a systemic approach to interventions. Schools and 
districts are increasingly organizing specific strategies or interventions into a continuum of supports 
that is based on the needs of the students and the intensity and the duration of the interventions or 
supports. One model of this approach is the Massachusetts Tiered System of Support (MTSS), which 
outlines a blueprint that is responsive to the academic and nonacademic needs of all students. MTSS 
offers a structure to provide students with access to high-quality core educational experiences in a 
safe and supportive learning environment and a continuum of services that provides increasing levels 
of support. The three tiers of MTSS are as follows: 

■ Tier I interventions and supports are the general education environments that are safe and 
supportive, the curriculum is rich and rigorous, and the instruction is designed to meet the 
variability of learners. 

■ Tier II interventions and supports are moderately intensive and are targeted toward small groups of 
students with similar needs who would benefit from similar interventions or supports (and 
sometimes individual students). 

■ Tier III interventions and supports are the most intensive and are provided to individual students 
with the highest levels of need. 

Exhibit 9 is a graphical depiction of the flexible tiers for providing interventions 

and supports within MTSS. For more information on MTSS, see http://www.doe.mass.edu/mtss . 
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Exhibit 9. Flexible Tiered Approach to Student Support 



When adopting or adapting such a model for keeping students on track, schools and/or districts may 

consider the following: 

■ Clearly communicate the purpose of and define the tiers within the model so that stakeholders— 
including administrators, educators, families, specialists, school support staff, outside agencies, and 
students— easily understand the tiers and their flexibility . 2 

■ Establish a protocol that enables students to move through the tiers seamlessly and efficiently as 
needs are identified and then change. 

■ Ensure that more intensive interventions and supports do not result in a label 3 for students but 
result in actionable information that leads to appropriate support. 


2 For example, a student assigned intensive 1:1 academic support to reach a certain skill level, might be moved to a small group 
intervention over time. Eventually, that same student may not need either support, but, when he/she moves to a new grade or 

school, might be reassigned to more intensive or targeted supports. 
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It is assumed that those schools or districts using a tiered approach enables all students to have access 
to Tier I instructions and supports: broad, school, and/or districtwide changes and supports. Students in 
need also may receive Tier II or Tier III interventions and supports, or both, based on the assessment of 
need determined by a variety of factors and the team. In Step 5, the team monitors student progress 
and adjusts student assignments to interventions and supports as well as their movement through tiers, 
as needed. These features help schools and districts coordinate services and closely track the 
participation of individual students in interventions and supports and their responses to those supports. 
Important Note: Schools and districts that are not using a tiered model of support can still make good 
use of EWIS and monitoring data with their own systems of support for students in need. 


Creating an Inventory of Available Interventions and Supports 



Notes from the Field 


We started out by doing a 
comprehensive inventory of all 
the interventions we offered at 
all grade levels. When we 
reviewed this list we saw right 
away that we had virtually 
nothing for middle school, and 
that our interventions were being 
thought about and delivered in 
silos by ELL, SPED, etc., rather 
than addressing the broader 
needs of students. 



Before providing interventions and supports to meet student 
needs, the team needs to identify what is already available 
and better understand the needs those interventions and 
supports are intended to address. By creating an inventory 
of existing programs, services, interventions, and supports, 
along with the personnel who will provide them in the 
school, the district, and, if possible, the community, the 
inventory is a valuable resource on which the team draws as 
it identifies students in need. Some districts and schools 
may already have information about available interventions 
and supports and may need to locate and combine this 
information from a variety of departments and other 
sources. Tool 14 provides a sample way to organize this 
information. 

When conducting an inventory of existing interventions and 
supports, the team reviews available interventions and 
supports (see Tool 15). The following questions help teams 


to organize and review these interventions and/or supports: 


■ What is the nature and the purpose of each intervention and/or support? What grades or age 
groups does each intervention serve? 


3 Teams should ensure that students are not designated as Tier I, II, or III students; rather the interventions and 
supports are Tier I, II, or III. For example, students may receive universal supports (Tier I) for most aspects of the 
their education, at the same time as receiving Tier II supports for math and Tier III supports for mental health 
services. 
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■ Which student needs is each intervention and/or 
support primarily designed to address (e.g., academic, 
learning skills, social, emotional, attitudes and beliefs, 
behavior, health, student disability and/or learning 
profile, language proficiency)? 

■ What is the intensity of each intervention or support 
(e.g. Tier I, II, III)? 

■ What is the duration of each intervention or support? 

■ Where and when is each intervention and/or support 
being offered (e.g., which classrooms, schools, locations 
within the district or the community)? 

■ What is the current capacity of teachers, 
administrators, or specialists for implementing each 
intervention or support for identified youth? 

■ Are any of the students who have been identified as 
being at risk already participating in these interventions 
or supports provided outside the school day? 

■ Are the interventions and supports evidence based and known to be effective? For which profile of 
students are they effective? What is the evidence? 

■ Are there universal screenings and assessments in place that are valid and reliable predictors of 
future performance that are (or could be) used to provide interventions? 

■ What are the gaps in the currently available interventions and/or supports? 

Providing Interventions and Supports 

After the inventory is developed, the team needs to prepare to take action. The team considers how 
student needs identified in Step 3 can be addressed within the continuum of existing interventions and 
supports. The team may want to begin by identifying those programs that both meet identified needs 
and are currently available on-site at the school. For some teams, the inventory of interventions and 
supports will reveal that there are few interventions and supports available that can help meet 
identified student needs. If this is the case, the team needs to determine the nature, the level, and the 
type of new interventions and supports that are needed. For a listing of resources for Targeted and 
Schoolwide Interventions and Supports, see Appendix D. This list provides information on intervention 
types, the appropriateness of interventions by school levels, and the level of research review and rigor 
of evidence for listed interventions. 

Often, the team begins by focusing on Tier I instruction and supports that can serve the greatest number 
of students in need, before assigning more intensive interventions or supports to groups of students and 
individual students. For instance, should a school adopt a new instructional approach, such as project- 



Notes from the Field 


We developed standard 
protocols to use across the 
schools to assign students to 
support tiers in a consistent 
fashion based on relevant and 
predictive data. 
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based learning, to help engage students and align more effectively with students' learning styles? Are 
there school or district policies and practices that will help address student learning needs, such as new 
behavior management policies? Step 4 relies heavily on the data collected during Steps 2 and 3 to 
inform action, but team members are ultimately charged with using their professional judgment to 
evaluate data and upon which recommend specific interventions and supports. 16 Remember, teams can 
always adjust and add interventions and supports throughout the year. 

To help track student progress, it is recommended that the team document any small group or 
individual intervention and support assignments. As students receive interventions and supports, the 
team can monitor student progress and adjust the intervention or support on the basis of this 
information (see Step 5). 


To ensure that each intervention or support is appropriate and effective, the team must: 

■ Secure leadership buy-in and support for intervention and support strategies to assist students. 

■ Collaborate with and secure buy-in and time from classroom teachers, school counselors, special 
educators, and other specialists to learn how to design, implement, and oversee interventions and 
supports. 

■ Ensure that planned interventions and supports will provide sufficient intensity and that 
participation or attendance in interventions and supports is documented. 

■ Secure buy-in from students and families (as appropriate). If the school is implementing Individual 
Learning Plans (ILPs) for students the ILP process is a place to discuss possible intervention and 
supports. These interventions and supports can also be documented in the ILP instrument. 


■ Devise a plan for tracking individual student 
responses to the interventions or supports. For 
example, use monitoring indicators to identify 
whether students in interventions continue to be 
flagged (Step 5). 17 

District Role in Step 4 

The perspective of the district allows longer-term 
solutions and strategies across schools. The district can do 
the following to support its teams in identifying 
appropriate interventions and supports: 

■ Identify common needs of students across multiple 
schools. 

■ Pinpoint districtwide solutions for common needs 
across schools. 



Notes from the Field 


It’s not always about the student 
when they are high risk. 

We used EWIS data to go to 
some teachers with high failure 
rates and ask them to tell us 
about what's happening in their 
classrooms. We were able to 
suggest shifts in instructional 
practices, use more formative 
assessments and embed 
instructional coaches in 
classrooms to offer more support 
for some teachers. 

~ 
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■ Provide solutions for gaps in available interventions and supports. 

■ Allocate and/or reallocate resources on the basis of identified needs within specific schools. 

■ Support interventions to improve student readiness for key transitions (e.g., elementary school to 
middle school, middle school to high school, summer bridge programs). 

Districts also have a role to play in identifying and promoting the use of promising intervention and 
support programs. By working with and as part of teams, district personnel can play key roles in 
identifying promising interventions and supports and share effective practices across schools. 

Box 3. Examples of District Activities 


The following are some specific activities that districts can use to support schools in Step 4: 

■ Develop policies, practices, and procedures for implementing a tiered system of support. 

■ Ensure that parents receive regular updates of their children's interventions, goals, and 
progress. 

■ Demonstrate awareness and sensitivity to cultural, linguistic, and other aspects of family 
diversity. 

■ Engage community partners and providers with the school and families to provide access to 
social services and health, social, recreational, and supplemental educational services. 

■ Promote equity by distinguishing among the academic and/or nonacademic needs of individual 
schools and populations and allocating adequate resources to the schools and students with 
the greatest needs. 

■ Implement a review process to determine the cost-effectiveness of its programs, initiatives, and 
activities as it relates to student achievement. 

■ Actively seek ways to leverage resources and expand capacity through collaboration with 
external and community partners. 

■ Create a climate conducive to adult learning through effective communication, ongoing 
professional improvement, and joint responsibility for students' learning. 

■ Offer professional development programs and services based on district priorities, information 
about staff needs, student achievement data, and assessments of academic and/or 
nonacademic practices and programs at each school. 


Guiding Questions for Step 4 

■ What needs has the team identified for individual and groups of students? Schoolwide needs? 
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■ Do trends in the data identify the need for particular types of interventions and supports (e.g., 
professional development for teachers on instructional strategies, transition supports, or 
opportunities for extended learning beyond the school day)? 


■ What interventions and supports are currently implemented in the school? In other locations in the 
district? How successful do they seem to be? What changes could be made in curriculum and 
instructional practices? What additional professional development is needed for teachers? 

■ What evidence based interventions and supports are available to support identified student needs? 

■ What policies and structural quality features (as opposed to specific programs) currently exist (or 
could be implemented) to support students' academic and nonacademic needs (e.g., flexible 
scheduling, behavioral support, or attendance and truancy policies)? 

■ Which interventions and supports can the team identify that, if implemented, would serve the 
greatest number of students? 

■ If a tiered model is not already in place, is this an approach our team wants to use to provide 
supports? Are there other ways to coordinate services and prioritize the allocation of resources? 

■ Do the demographic characteristics (e.g., disability status, economically disadvantaged status, or ELL 
status) of the students identified as being at risk inform intervention and support decisions? Should 
they? 
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Tool 13 - Step 4: Checklist 


Confirm Student Needs and Priorities: 

□ Confirm the list of individual and groups of students to which your team will assign 
interventions and supports 

□ Identify common needs across individuals and/or across groups 

□ Identify any schoolwide needs your team plans to address 

Inventory of Interventions and Supports: 

□ Create an Inventory of current interventions and supports 

□ Compare the list of interventions and supports to identified student and schoolwide 
needs 

□ Identify any gaps between identified needs and available interventions 

□ Determine what new/additional interventions (across all three tiers) may need to be 
developed, offered, or identified 

Matching Needs with Support and Interventions: 

□ Determine what approach your team will use to assign interventions and supports (e.g. 
How do you already assign interventions and supports? Do you have a tiered model of 
support in place? If not, is this an approach the team wants to explore?) 

□ Determine the broad policies and structural quality features (as opposed to specific 
programs) that could be implemented to support students' academic and nonacademic 
needs 

□ Determine which individual or group interventions and supports, if implemented, 
would serve the greatest number of students 

Assigning Interventions and Supports: 

□ Determine which interventions and supports your team will assign— and to which 
students 

□ Determine who will communicate with students, staff, and parents regarding these 
proposed supports 

□ Identify who will oversee/manage interventions that are assigned 

□ Determine what data your team will collect regarding student participation and progress 
in interventions and supports 

□ Determine when/who will be responsible for providing these data to the team 
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Tool 14 - Step 4: Inventory of Interventions and Supports 


Directions: Use the table below to develop a list (or inventory) of available supports and interventions in the school, the district, and the 
community that are available to meet students' needs. Describe the focus of each intervention and support in column B. In column C, indicate 
the level of intensity - universal (U), supplemental (S), or intensive (I) - for which this applies within a tiered system of supports. Complete 
columns D-l to provide additional information and detail about each intervention or support to assist your team in matching the appropriate 
interventions and supports to the needs of students on an individual, group, or schoolwide basis. 


Interventions and Supports Inventory 


A 

B 

c 

D 

E 

F 

G 

H 

1 

Name of 

Intervention or 
Support 

Description 

Intensity Level 
(U, S, or, 1) 

rype of Support 

(e.g., academic, 
aehavioral) 

Specific Skill 

Area the Support 
/ Intervention 
Will Address 

(e.g., 

ohonological 
awareness, self- 
regulation) 

Format 

(schoolwide, 
class, small 
group, 
individual) 

Age/Grades 

target Population 

(e.g., student need) 

Frequency and 
Duration 

EX: Academic 

Tutoring Mass Vol 

Scholars 

Volunteer tutors from 
local university 

S 

Academic, social- 

emotional 

Math 

Individual / 
small group 

2nd -4 th 

Students failing l st -3 rd 
grade math 

benchmark 

assessments. 

2 days / week each 
quarter 
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A 

B 

C 

D 

E 

F 

G 

H 

1 

Name of 

Intervention or 
Support 

Description 

Intensity Level 

(U, S, or, 1) 

rype of Support 

(e.g., academic, 
aehavioral) 

Specific Skill 

Area the Support 
! Intervention 
Will Address 

(e.g., 

ohonological 
awareness, self- 
regulation) 

Format 

(schoolwide, 
class, small 
group, 
individual) 

Age/Grades 

Target Population 

(e.g., student need) 

Frequency and 
Duration 
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Tool 15 - Step 4: Interventions and Supports Review 


Directions: Use the table below to review and monitor each available intervention or support and use this information to (1) improve the team's 
ability to match appropriate supports and interventions to the needs of students, (2) monitor the impact of interventions and supports, and (3) 
conduct a gap analysis of available supports and interventions to meet the needs of students. 

List and describe the intervention or support in columns A and B. In column C, indicate the level of support for which this applies within a tiered 
system of supports. Use columns D-E to provide further information about each intervention or support. Use columns F-H as the team monitors 
interventions and supports to track information about cost, evidence of improvement, and notes about the type of student needs that the 
intervention or support meets. 


A 

B 

C 

D 

E 

F 

G 

H 

Name of Intervention 
or Support 

Description 

Intensity Level 

(U, S, or, 1) 

Fype of Support 

(e.g., academic, 
behavioral) 

Age/Grades 

Cost Per 

Student 

Metrics to Track 

Evidence of 
Improvement 

Notes: Fit Between Support 
and Student Needs 

EX: Academic Tutoring: 

UMass Volunteer Scholars 

Volunteer tutors from local 
university 

S 

Academic, social- 

emotional 

Gr 2-4 

$50 

Tutor - teacher 

communication log, 
quarterly benchmark 
assessments, teacher 

progress reports 

Many identified students do not 
speak English. Tutors are bi- 
lingual and speak the primary 
language of most students at 

the school. 
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A 

B 

C 

D 

E 

F 

G 

H 

Name of Intervention 
or Support 

Description 

Intensity Level 

(U, S, or, 1) 

Fype of Support 

(e.g., academic, 
behavioral) 

Age/Grades 

Cost Per 

Student 

Metrics to Track 

Evidence of 
Improvement 

Notes: Fit Between Support 
and Student Needs 
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A 

B 

C 

D 

E 

F 

G 

H 

Name of Intervention 
or Support 

Description 

Intensity Level 

(U, S, or, 1) 

Fype of Support 

(e.g., academic, 
behavioral) 

Age/Grades 

Cost Per 

Student 

Metrics to Track 

Evidence of 
Improvement 

Notes: Fit Between Support 
and Student Needs 
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Tool 16 - Step 4: Review Your Inventory of Interventions and Supports 


Directions: Now that you have completed your Interventions and Supports Inventory, explore the following questions. 

Reviewing Your Inventory - Guiding Questions: 

■ Are there any interventions or supports that need to be added? Add any additional interventions or supports that are in place (e.g., districts 
supports, informal supports.) 

■ Thinking of students varying needs for intervention intensity (e.g. tiers) are there any areas in which you may not have adequate supports? 

■ Thinking of the early warning data for your students and your analysis, are there any supports that you may still need to develop? What are 
they? 


Assigning Supports - Guiding Questions: 


■ What is the range of interventions and supports we have available? 

■ What gaps can we see between student needs and available supports and interventions? 

■ What do we know about the proven effectiveness of these supports and their suitability for identified student needs? 

■ Which interventions or supports, if implemented, could serve a group of students? Which would serve the greatest number of 
students? 

■ What changes to grade- or school-level policies, procedures, scheduling, professional development, or curriculum could we make 
to meet common student needs? 
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Tool 17 - Step 4: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 4. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 

Key Tasks: 

• Confirm the priorities for student needs and support goals - particularly common needs across individual students, groups of students, 
and/or all students within the school/district. 

• Determine how decisions will be made about providing supports (e.g., tiered system of support). 

• Create and/or build on existing inventory of student supports (e.g., complete the interventions and supports inventory). 

• Review how student needs could be addressed through existing interventions and supports and identify which interventions and 
supports would serve the greatest number of students. 

• Identify any gaps and determine whether additional interventions and supports are needed. 

• Determine any school /district-wide policies and procedures and/or other broad curricular changes that could help address common 
needs. 

• Communicate with teachers, specialists, students, and families about the assigned interventions and supports. 

• Assign and document assigned interventions and supports. 


Objective or Task 

What do we have in place? 
What expertise and/or 
resources do we already 
have? 

What additional actions 

and resources are needed? 

Who will be responsible? 

What is the time frame? 
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Step 5: Monitor Students and Interventions 


What You Need for Step 5 

■ The most up-to-date student data from the district's or the school's student information or other 
database or system that will be used to monitor student progress 

■ A list of students previously identified as at-risk and their assigned interventions 

■ Access to additional student data sources for students who are not showing progress 

■ An inventory of interventions and supports 

■ Tool 14- Interventions and Supports Inventory 

■ Tool 15 - Interventions and Supports Review 


Description of Step 5 


Monitoring indicators are both research based and locally 
determined sources of student data available during the school 
year that are used to flag students in need. Indicators such as 
attendance, academic performance, and behavior have been 
identified in research as strong monitoring indicators. Monitoring 
indicators can complement the use of EWIS data by enabling teams 
to track student needs and students' responses to interventions or 
supports at multiple times throughout a single school year. 

Tool 19 in Step 5 lists a set of commonly used monitoring 
indicators by age group and grade. Each data source and monitoring 
indicator threshold, or the point at which a student is flagged as in 
need, is determined when a research base is available. These 
indicators are shown to be research supported and marked as such in 
the fourth column of Tool 19. In instances where an indicator has 
been used in practice but there is no consensus on a threshold, or 
if the threshold is dependent on the instrument used (e.g., 



Build Off Your Foundation 


• How do you monitor student 
progress? 

■ Which sources of data do you 
use? Which data systems do you 
use to organize this information? 

■ How can you integrate EWIS 
data into your current data 
systems? 

■ How can you align your review of 
monitoring indicators with the 
timing and data work being done 
by existing groups? 



benchmark assessment), then the fifth column in Tool 19 shows that the indicator must be locally 


determined by the school or district. 


It is recommended that teams decide before— or early on in the school year— which student data 
sources they will use to examine student risk throughout the year, set thresholds that will "flag" 
students as being at-risk, and set up a data system that can be used to monitor students. 
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In Step 5 teams assess student progress, improve the effectiveness of interventions and supports, and 
identify any new students that are flagged as at-risk. The team uses monitoring indicators data to see 
whether students are beginning to get back on track (i.e., not flagged for a monitoring indicator), or 
whether they are still showing (or newly showing) signs of risk. For students flagged as at-risk, the team 
will assign, re-assign, and/or adjust interventions and supports, as needed. Monitoring flags are also 
used to observe any students, who were not originally identified as EWIS high or moderate risk, who 
may be showing symptoms of need. 

During Step 5 the team generates and reviews up-to-date information on student risk. As part of 
monitoring, teams revisit Steps 3 and 4, Explore Underlying Causes of Risk and Take Action so that teams 
can interpret the meaning of risk flags, and adjust or match students with new or different interventions 
and supports. Step 5 also helps the team examine trends in student responses to assigned interventions 
that may allow a district or a school to assess the effectiveness of certain interventions and supports for 
certain students or groups of students. Finally, the regular monitoring of students and schoolwide 
interventions and supports allows the team to engage in planning for summer and other extended day 
programs and supports using data. 

Anticipated Outcomes for Step 5 

The following outcomes are anticipated for Step 5: 

■ Identification of students who are newly flagged as at-risk, or who are continuing to be flagged as 
at-risk 

■ Knowledge about individual student, subgroup, and 
whole school progress in addressing student risk 

■ Identification of gaps in the available interventions and 
supports for students, recommendations for new 
intervention strategies, and prioritization of new 
interventions and supports based on monitoring flags 

■ Increased awareness of how well interventions and 
supports are working, and which interventions and 
supports appear to be most effective in helping 
students get back on track 

■ Enhanced understanding by appropriate stakeholders 
about student needs, the impact of existing 
interventions and supports, and the need for additional 
interventions and supports, if applicable 



Notes from the Field 


We met once or twice a month to 
look at data so we could monitor 
student progress. We spent a lot 
of time interpreting the data 
because there were surprises. 
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Preparing to Monitor Students and Interventions 


During Step 1: Get Organized, teams prepare to use early warning data, including research-based and 
locally determined monitoring indicator data before or during the early part of the school year. These 
steps need to be completed prior to monitoring students and interventions: 

■ Select which student data sources the team will use to monitor student risk throughout the year. 

■ Establish thresholds to "flag" students as being at risk using these data sources (e.g., students who 
miss 10 percent or more of school days). 

■ Establish a system or tool your team will use to monitor students' early warning data. 

■ If using a data system or tool, make sure that at least one member of the team has proficiency in 
importing, updating, and generating reports using the 
monitoring indicators database system or tool. 

■ If using a data system or tool, establish a data import 
and review schedule for analyzing data and reports with 
up-to-date student monitoring indicators data 
throughout the school year. 

■ Verify that all members of the team understand the data 
sources for monitoring student progress and the various 
monitoring indicator reports. 

Teams can select from one or more monitoring indicators. 

As previously mentioned, a monitoring indicator draws 
from readily available data that can be observed at many 
points during the school year. To review or revise your 
monitoring indicators, please see Tool 19 below. It is 
recommended that the team identify or establish a system 
before, or early on in the school year, to manage the data 
that will be used to monitor student risk flags. The team has 
various options for managing these data. For instance, districts and schools may wish to use their own 
data systems for monitoring students. Or, districts and schools may choose to other available systems 
for monitoring students, such as AIR's National High School Center EWS High School Tool or EWS Middle 
School Tool 18 which is available for free or other commercially available monitoring indicator tools. 

Data Import and Monitoring Schedule 

A key aspect of monitoring is the timely access and review of student data to determine which students 
are "flagged" as at-risk, and for which indicators students are "flagged". The pre-set thresholds 
associated with monitoring indicator data sources are grounded in research that specifies particular 
timeframes for these thresholds. For example, the research on the relationship between attendance and 
student risk found that students who missed more than 10% of instructional days during any given 



Notes from the Field 

By monitoring students we found 
out that the intervention worked 
for 1 6 out of 18 students - and 
that they showed increased 
literacy scores and scored higher 
on other benchmarks. The data 
also showed that giving these 
students a fresh new start did 
nothing to affect their attendance - 
for some it even got worse . . .For 
those who weren't absent and 
who were able to work directly 
with a literacy coach, they we 
able to move much more rapidly 
and get up to grade level. 

1 
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marking period were more likely to drop out of high school. 
Given these research-based timeframes, typically, teams 
examine monitoring indicators following each grading 
period. In addition, many teams examine student 
attendance after the first 20-30 days of school— another 
timeframe identified through the research. 

The team and the data designee should work together to 
establish a data import and review schedule for analyzing 
data and reports with up-to-date student monitoring 
indicators data throughout the year. It is important to leave 
sufficient time between the end of each grading period and 
the scheduled team meetings so the team's data designee 
has sufficient time to import, analyze and generate reports 
for the team to review to track student progress. 

Monitoring Student Progress in Interventions 
and Supports 



Notes from the Field 


EWIS data is a good starting 
point, but once the school year 
starts, that data is changing - 
i.e., how a student is once the 
school year starts might be 
different. We found that it was 
really helpful to keep looking at 
how students were performing. 




The team can use the following strategies to monitor student progress: 


■ Have a Data Designee. Each team should designate at least one member who has data access, and 
who has the proficiency and the time to take primary responsibility for importing, updating and 


generating grading period reports that will be used for monitoring students throughout the year. In 


Notes from the Field 

We have done a complete 180 
degree turn in how we are using 
data this year. We were using 
data in an autopsy format all 
along - the data was rich, but we | 
were waiting for State 
Assessment results to come out 
to say "What did we do wrong?" 
Now, we see student status, 
their progress, and adjust 
student supports RIGHT NOW 
instead of waiting until after kids 
have dropped out or repeated a 
grade. 


addition, it is recommended that the data designee take the 
time to help all team members understand the data sources 
being used to monitor student progress, and how to 
accurately interpret monitoring indicator reports. 

■ Generate up-to-date information on risk. A team 
needs current monitoring indicators data before meeting to 
track student progress. In Step 1, the team establishes a 
system to use monitoring indicators and establishes a 
schedule and a person responsible for importing student 
data at these specific time points e.g. attendance and 
course performance data after each grading period. 

■ Review current monitoring indicators data to identify 
students who are flagged as at-risk. For the monitoring of 
individual students and groups of students, the review of data 
should focus on two sets of students: (1) students who were 
previously at risk and assigned to one or more interventions or 
supports, and (2) students who were NOT previously identified 
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as at risk, but are now showing risk flags. The review of data for previously identified students should 
determine student progress (i.e., Have students improved or do they continue to be flagged as at-risk?). 
For example, if a group of students identified as being at high risk for failing their 9 th grade classes were 
assigned to an afterschool tutoring program, do first-term grades for these particular students show any 
improvement? See Tool 20 in Step 5 for a worksheet on exploring your monitoring indicators data. 

Many teams review monitoring indicator data for all students each month or following each grading 
period. This monitoring should be done as part of the team's routine review of student data and 
incorporated as a regular item on the team's agenda. How and how often this monitoring is 
conducted depends on the level and the nature of interventions and supports that were 
implemented in Step 4. Keep in mind that students participating in Tier 3 interventions should 
monitored frequently. 

■ Identify student needs that are not being met— and explore underlying causes for signs of risk. 

Students who are receiving interventions and supports may continue to show signs that they are at risk 
for not meeting academic milestones. It is possible that their needs are not being addressed by the 
interventions or supports for any number of reasons. Regular monitoring allows these issues to be 
identified quickly. The team may need to collect additional information— particularly data related to 
student participation in assigned interventions and supports to understand why an intervention or a 
support is not working. After Step 5, the team is likely to revisit Step 3 for students who are not showing 
progress or who are newly flagged at risk to identify the nature of unmet student needs, and likely 
underlying causes of risk. 

■ Adjust or assign new or additional interventions or supports to meet student needs. The team may 
need to identify new or additional interventions or supports that are not currently available to 
students identified as being at risk. To accomplish this, the team should revisit Step 4 activities (i.e., 
conduct a search and update the list of available interventions and supports that may serve 
identified student needs, and assign the appropriate interventions or supports). In some cases, the 
team is able to find existing interventions and supports that meet student needs; in other cases, 
new interventions and supports will be needed or broader, schoolwide changes will be 
implemented. Any new interventions and supports should be tracked and captured on the 
interventions and supports inventory (created in Step 4). 

■ Communicate with educators, specialists, students and families. As noted in Step 4, teams will 
want to communicate with educators, specialists, students, and their families to best support 
students who are at risk for not meeting academic milestones. Educators, specialists, and families 
should be given guidance on how they can best support these students and be informed when 
students appear to make improvements and when there is a lack of improvement or further decline. 

■ Review current monitoring indicators data to track schoolwide interventions. For monitoring 
districtwide and schoolwide interventions and supports, the team may want to identify which 
indicators or other data sources to track schoolwide progress. For instance, if a new mathematics 
curriculum has been introduced, the team may want to examine midterm mathematics grades. 
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■ Examine the effectiveness of the interventions or 
supports. In addition to tracking the progress of 
individual students and schoolwide interventions, Step 
5 also encourages the team to better understand the 
effectiveness of particular interventions and supports. 

Long term, this process improves the team's 
understanding of which interventions and supports are 
best at addressing particular student needs and enables 
the team to strengthen the process of matching 
students to the right interventions and supports (see 
the district role for Step 5). 

District Role in Step 5 

The monitoring of individual students and groups of 
students occurs at the school level, but the district can play 
a key role in enhancing the availability of appropriate 
interventions and supports. For example, a district can 
monitor specific interventions and supports in which 

students participate and identify the scenarios in which 
these interventions and supports seem to work to get 
students back on track. The district also may examine 
whether a specific intervention or support works well 
for certain groups of students (e.g., students who are 
failing ELA classes or ELL students at risk). In cases 
where an intervention or a support does not seem to be 
working, the district can examine whether fidelity of 
implementation is the issue or if the intervention or the 
support needs to be dropped from the array of student 
interventions and supports. 

By collecting districtwide information on interventions 
and supports implemented in its various schools, a 
district is in a unique position to understand which 
interventions and supports are most effective at meeting 
the needs of particular students and subsequently 
allocate the resources (e.g., staff and/or funds) on the basis of student and school needs. 

Guiding Questions for Step 5 

■ Do students who are identified as at-risk by EWIS data in the beginning of the year continue to be 
identified as at-risk based on monitoring indicators? 


Implementation Tip : 

Planning Supports for the Summer 

The team needs to meet multiple times 
throughout the year to track student progress 
and make adjustments in interventions and 
supports. In the spring, the team can consider 
which students may need or benefit from 
interventions and supports during the 
summer. The team can then identify which 
students will need summer support, devise a 
plan for how to prioritize student needs, 
assess available summer programs and/or 
supports, and consider resources. The team 
may want to invite administrators for summer 
school programs to meetings to help make 
these assignments. 



Notes from the Field 


We plan to look over the 
summer at our high risk kids who 
did great and figure out what 
circumstances helped them, 
what worked? 
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■ What are the most prevalent monitoring indicators for which students are flagged (e.g., attendance, 
course failures, behavior)? 


■ Are students (at the individual, group, and whole-school levels) who are participating in 
interventions and supports showing improvement? If so, will these students need to continue 
participating in the interventions or supports to maintain their progress and improved outcomes? 

■ Are students who are participating in interventions or supports continuing to be identified as at- 
risk? 

■ What are the likely reasons for the continued risk for students not showing progress? 

• Did the team adequately assess the underlying reasons for student risk during its initial review 
(Step 3)? 

• Are students fully participating in the assigned intervention(s) or support(s)? Are there problems 
with how the interventions or supports are being implemented? 

• Are the assigned interventions or supports appropriate for the nature and the severity of need? 

• Should the frequency and/or intensity of the interventions or supports be increased? 

• Have new needs or issues emerged that need to be addressed? 

• What additional information needs to be gathered (Step 3) to achieve a better fit between each 
intervention or support and student needs (Step 4)? 

■ Are any new students flagged as at-risk that had an EWIS student risk level of low or moderate at 
the beginning of the school year? What are the likely underlying causes of risk for these students? 
Which additional sources of data should we collect to better understand their needs? What 
interventions and supports are appropriate for meeting their needs? 

■ With whom and how should our team communicate about student progress or concerns to 
maximize support for students in need (e.g., teachers, specialists, parents, students, administrators, 
community agency staff)? 

■ Are organizational or structural changes needed in the school or the district to support students? 

■ As interventions and supports are implemented, what happens to the nature, the level, and the 
number of students with identified needs? Has the number of students in need for each indicator 
changed since the new interventions and supports have been put in place? 

■ Are resources (e.g., staff time, funds, transportation) sufficient to implement the desired 
interventions and supports? If not, how might additional resources be identified? 

■ What are we learning about the effectiveness of the interventions the supports? 

• Which interventions and supports seem to consistently get students back on track? 

• What interventions and supports seem to work for which students? 

• What interventions and supports do not seem to work for students? How might these be 
improved? Should these be eliminated? 
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Tool 18 - Step 5: Checklist 


Generate up-to-date information on student risk: 

□ Confirm data import schedule and individual responsible for importing these data and 
generating reports for the team prior to the meeting 

□ Confirm the team meeting schedule allows for sufficient time for reviewing data 

□ Generate monitoring indicator reports for all students 

Review current monitoring indicator data to flag students: 

□ Review data for students who were previously flagged to assess progress (e.g., Does a 
student continue to be flagged?) 

□ Review data to see if new students are flagged 

□ Review monitoring indicator data to track schoolwide progress toward broader school 
interventions and/or improvement goals 

Revisit Step 3: Explore underlying causes and identify student needs: 

□ Determine additional data sources that are needed to identify underlying causes for 
students' being flagged 

□ Interpret additional student data and identify student needs 

Revisit Step 4: Assign or adjust interventions and supports: 

□ Determine which interventions and supports your team will assign— and to which 
students (individuals, groups, or whole school) 

□ Determine who will communicate with students, staff, and parents regarding these 
proposed supports 
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Tool 19 - Step 5: Your Selected Monitoring Indicators by Grade Level 

Directions: Use the table below to document the monitoring indicators that were selected for use throughout the school year. Each grade level 
includes attendance, behavior, and academic performance potential monitoring indicators flags students that are on or off track. In the "Using?" 
column, check of which monitoring indicators your school or district is using this year and note the data source and selected threshold for the 
monitoring indicator. 


Academic 

Grade 

Potential 

Monitoring 

Indicator 

Threshold 

Research 

Locally 

Using? 

Notes - existing data source, 

milestone 

Supported 

Determined 

selected threshold, etc. 

Early Elementary 



Proficient or higher 
on grade 3 ELA State Assessment 

1 

Attendance 

Locally defined 


X 



Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 




2 

Attendance 

Locally defined 


X 



Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 




3 

Attendance 

Locally defined 


X 



Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 





Additional notes about monitoring indicators for grades 1, 2, and 3: 
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Academic 

milestone 

Grade 

Potential 

Monitoring 

Indicator 

Threshold 

Research 

Supported 

Locally 

Determined 

Using? 

Notes - existing data source, 
selected threshold, etc. 

Late Elementary 



Proficient or higher on grade 6 

ELA and mathematics State Assessment 

4 

Attendance 

20% or more school days missed 

X 




Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 




5 

Attendance 

20% or more school days missed 

X 




Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Course 

performance 

Failing grade in mathematics and/or ELA 

X 




Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 




6 

Attendance 

20% or more school days missed 

X 




Behavior 

Locally defined behavior grade, office 
discipline referrals 


X 



Course 

performance 

Failing grade in mathematics and/or ELA 

X 




Academic 

performance 

Locally defined performance levels on 
Benchmark assessments 

X 





Additional notes about monitoring indicators for grades 4, 5, and 6: 
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Academic 

milestone 

Grade 

Potential 

Monitoring 

Indicator 

Threshold 

Research 

Supported 

Locally 

Determined 

Using? 

Notes - existing data source, 
selected threshold, etc. 

Middle School 



Passing grades in 
all grade 9 courses 

7 

Attendance 

20% or more school days missed 

X 




Course 

performance 

Failing grade in mathematics and/or ELA 

X 




Behavior 

Suspensions, office discipline referrals 


X 



8 

Attendance 

20% or more school days 

X 




Course 

performance 

Failing grade in mathematics and/or ELA 

X 




Behavior 

Suspensions, office discipline referrals 


X 



9 

Attendance 

10% or more school days missed 

X 




Course 

performance 

Failing one or more grading period 

X 




Credits 

Enough credits for promotion to next 
grade 

X 




Behavior 

Suspensions, office discipline referrals 


X 




Additional notes about monitoring indicators for grades 7 , 8, and 9: 
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Academic 

milestone 

Grade 

Potential 

Monitoring 

Indicator 

Threshold 

Research 

Supported 

Locally 

Validated 

Using? 

Notes - existing data source, 
selected threshold, etc. 

High School 




10 

Attendance 

10% or more school days missed 

X 






Course 

performance 

Failing one or more grading period 

X 






Behavior 

Suspensions, office discipline referrals 


X 



o 

'■+-> 

D 

"O 

11 

Attendance 

Locally defined percentage of school 
days missed 


X 



03 

i_ 

CUD 

~o 

O 


Course 

performance 

Locally defined number of course 
failures 


X 



u 

CO 


Behavior 

Suspensions, office discipline referrals 


X 



WD 

ir 

12 

Attendance 

Locally defined percentage of school 
days missed 


X 





Course 

performance 

Locally defined number of course 
failures 


X 





Behavior 

Suspensions, office discipline referrals 


X 




Additional notes about monitoring indicators for grades 10, 11, and 12: 
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Tool 20 - Step 5: Early Warning Data Exploration Summary Chart for Monitoring Indicators 

Directions: Use this chart to take notes as you examine your monitoring indicators. 



Key Findings 

Areas of Concern 

Big Picture: Main Categories - Summarize your big picture monitoring indicator findings. 

■ # or % of students meeting or exceeding the 

threshold (i.e., flagged as at-risk by monitoring 
indicators) 



■ By grade 

■ By school 



By student groups and characteristics: 

■ Gender 

■ Race/ethnicity 

■ Low income 

■ ELL 

■ FELL 

■ SPED 

■ High Needs 

■ EWIS High Risk 

■ EWIS Moderate Risk 

■ EWIS Low Risk 



By types of monitoring indicators students for 
which students are flagged: 

■ Attendance 

■ Behavior 

■ State Assessment performance 

■ Course performance 

■ Other 
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Key Findings 

Areas of Concern 

Dig Deeper: Subgroups/Sub-Categories - Identify one student group, school, or grade, of concern to explore further based on the students 
flagged for monitoring indicators (e.g., middle school boys or students flagged due to attendance). This could be the same group you focused on 
in Step 2 when reviewing your EWIS data, or a new student group. Repeat this step for each targeted group of concern. 

By student groups within and across grades: 

■ Gender 

■ Race/ethnicity 

■ Low income 

■ ELL 

■ FELL 

■ SPED 

■ High Needs 

■ EWIS High Risk 

■ EWIS Moderate Risk 

■ EWIS Low Risk 



By types of monitoring indicators students for 
which students are flagged: 

■ Attendance 

■ Behavior 

■ Course performance 

■ State Assessment performance 

■ Other 
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Key Findings 

Areas of Concern 

Deep Dive: Individual Students - Identify individual students of concern to explore further. 

Shared characteristics of individual students of 



concern: 



o 

Gender 



o 

Race/ethnicity 



o 

Low income 



o 

ELL 



o 

FELL 



o 

SPED 



o 

High Needs 



o 

Attendance 



o 

Suspensions 



o 

Course subject Pass/ Fail 



o 

State Assessment performance 



o 

EWIS High Risk 



o 

EWIS Moderate Risk 



o 

EWIS Low Risk 



o 

Other characteristics: 
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Tool 21 - Step 5: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 5. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 

Key Tasks: 

• Generate up-to-date information on students (e.g., monitoring indicator data). 

• Review monitoring data to determine which students are flagged as "at risk". 

• Identify student needs that are not being met. 

• Examine monitoring indicator data (big picture, dig deeper, deep dive) 

• Collect additional information or data as needed to better understand underlying reasons for risk and needs (revisit Step 3). 

• Identify new or additional interventions to meet student needs (revisit Step 4). 

• Consider broader schoolwide/district-wide changes to meet student needs (revisit Step 4). 

• Examine the effectiveness of interventions for individuals and groups of students. 

• Communicate with teachers, specialists, students, and families to provide/adjust interventions and supports. 

• Conduct summer and end of year planning to support students who continue to show signs of risk. 


Objective or Task 

What do we have in place? 
What expertise and/or 
resources do we already 
have? 

What additional actions 

and resources are needed? 

Who will be responsible? 

What is the time frame? 
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Step 6: Refine the Early Warning Implementation Process 
What You Need for Step 6 

■ Documents describing the team's goals, mission, roles, and responsibilities 

■ List of monitoring indicators that were selected and used 

■ Data on the effectiveness of interventions and supports 

■ Tools 14 and 15 (provided in Step 4), the Interventions and Supports Inventory and Review 

Description of Step 6 

In Step 6, the team should reflect on the early warning implementation process. At a minimum, this 
reflection should occur on an annual basis, at the end of the school year or over the summer. The team 
discusses what has worked, what should be modified, and what should be replaced or eliminated. 

Where appropriate, any decisions about changes to the process should be supported by data and 
evidence and documented. Finally, the current team and other school and district leadership identifies 
which team members will continue to serve and who the new team members will be for the upcoming 
year. The district and the school create a plan for orienting new members and arranging for them to 
become trained in the early warning implementation process. 

Anticipated Outcomes for Step 6 

The following outcomes are anticipated for Step 6: 

■ A shared perception of the success and challenges of the early warning process 

■ Clear recommendations for improving the work of the team and steps in the process 

■ An established team for the upcoming school year, composed of members with a clear 
understanding of the process and their roles 

■ Documentation of any planned changes (e.g., data sources, tools, personnel, policies and 
procedures) 

Evaluating and Refining the Early Warning Implementation Process 

When evaluating the early warning implementation process, it is recommended that all aspects of the 
implementation process be considered, including how the team is organized and functions. The conversation 
can be guided by exploring what is working, what is not working, and what changes will be needed to 
improve the team's effectiveness and efficiency, as well as its ability to successfully achieve its goals for 
students. The team's examination can range from exploring the effectiveness of certain interventions to how 
the team can access data reports in a timely and an efficient manner. To ensure that the reflection meeting is 
productive, the team leader may want to ask all members to respond to a set of questions ahead of time (see 
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the guiding questions for Step 6). This allows the team facilitators to identify key areas of success and challenge 
and therefore focus the team's discussion in areas warranting the most change and improvement. 


District Role in Step 6 


The district-level team also should engage in periodic reflection. Similar to school-level teams, district- 
level teams should consider what is working and not working related to the goals and the tasks 
conducted by the district-level team and examine what the district could do to better support school- 
level efforts through leadership, district-level policies, and access to data and resources (e.g., staff time, 
transportation, support programs, and additional funds). It is suggested that a district-level 


representative either attend school-level reflection meetings or meet individually with a member of 


each school-level team to hear how the team is doing and 
how the district can help. 

Guiding Questions for Step 6 

The guiding questions for Step 6 encourage the team to 
reflect on each of the five previous steps. 

■ Step 1: Get Organized 

• What were the team's biggest successes? 

• What were the biggest challenges that the team 
faced? 

• What made the team's job easier? What changes 
could make the team's job even easier? 

• Who will continue to be on the team? (Note: It is 
recommended that some individuals overlap from 
year to year to ensure long-term continuity.) 

• Which roles will need to be filled for the upcoming 
year? Which other constituencies need to be 
represented on the team? 



Notes from the Field 


After a year of using EWIS we are 
integrating it into our work in 
several ways. Our Superintendent 
has decided to include it in monthly 
principal meetings - dedicating a 
portion of the meeting to using 
EWIS and doing a follow-up cycle. 
Building-based teams now include 
EWIS in leadership or student 
support team meetings, and next 
year we will have a multi-tiered 
team look at data together to come 
up with interventions, track student 
progress, and adjust interventions 
to improve outcomes. 


• What is important for new team members to know before deciding whether to join the team? 
What is important to know and be able to do when serving on the team? 


o What changes should be made in how the team organizes its work and functions as a 
team? How should the work be delegated or divided to ensure the fair distribution of 
responsibilities and tasks? 

o When and how frequently should the team meet? 
o How long should meetings be? 
o Do all members need to be at all meetings? 
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o What changes should be made in how the meetings are structured and run? 
o How much time should members expect to need to work outside meetings? 
o What changes should be made in how the team makes decisions? 


o What changes should be made in how the team communicates and shares information 
within the team, with other teams, and with the district? 

o What resources could support the team? 

o How could the team integrate its work more fully with existing teams, systems and 
procedures in the district/school? 

• What else should the team change to improve how the team organizes its work and functions? 

• What support is needed from the district to support the team? 

■ Step 2: Review EWIS data. 

• Were team members who were responsible for generating EWIS reports able to access the 
information easily? 

• Did the team feel proficient in using the reporting options? 

• Is there any aspect of using Edwin Analytics and/or EWIS that the team would like to learn more 
about? 

• Was the team clear about how to review EWIS data? Were the team members clear on the 
questions they hoped the data would help them answer? 

• Did the team members feel they had access to the right kinds and amount of data to answer 
their questions? Too much? Too little? 

• What challenges did team members face in trying to manage and understand the data they 
received? What kinds of training, guidance, or other support would the team members need to 
build their assessment literacy skills? 

• What additional questions about students would the team like to be able to ask and answer in 
the upcoming year? 

• What other changes would the team like to see in the amount, nature, format, frequency, and 
timing of the data to best understand the nature and the scope of student needs in their district, 
school, or cohort? 

• What else could the team do to improve this step? 

■ Step 3: Explore Underlying Causes 

• How did the team go about trying to discern the underlying cause for risk? 

• Did the team feel this approach was effective (i.e., did the team feel that it was able to identify 
the most important reasons why students were struggling)? 
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• How much time (e.g., the number of hours) did it take per student or group of students? How 
could this process go more quickly and smoothly? 


• Which sources of data were found to be most helpful in understanding student needs and the 
underlying causes for student risk? What other data would the team like to be able to access in 
the upcoming year? 

• Was the team satisfied with decisions about which students would be the focus of the team? 

• What else could the team do to improve this step? 

■ Step 4: Take Action 

• What challenges did the team face in trying to conduct the inventory of interventions and 
supports? What gaps does the team feel need to be filled in the upcoming year? 

• Did the team feel it was able to effectively match students with appropriate interventions or 
supports? How helpful were the EWIS data? The monitoring indicators data? Additional data? 
What else does the team believe could help with matching students to appropriate 
interventions or supports? 

• Was the team able to identify schoolwide areas of need based on early warning data? If not, 
how might the team consider broader schoolwide changes in response to EWIS data in the 
upcoming year? 

• What else could the team do to improve this step? 

■ Step 5: Monitor students and interventions. 

• How frequently did the team monitor student progress? Does the team feel it should be 
monitoring more often? Less often? With the same frequency? 

• What data did the team rely on most to assess student progress? What additional data does the 
team feel should be included next year? 

• Are the monitoring flag thresholds appropriate? Should they be adjusted? 

• What other sources of data did the team use to determine the effectiveness of interventions 
and supports in meeting student needs? Are there any data sources that the team wants to 
track more systematically in the upcoming year (e.g., participation or attendance in 
interventions or supports)? 

• Were the team members satisfied with the data system they used for the monitoring student 
progress? 

• By whom and how were interventions and supports managed for students and groups of 
students? Does the team believe that the school should create any new systems or structures that 
can help manage student interventions and supports? Could these changes help with the 
monitoring of student progress? 
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• Was the team able to develop any systems or approaches for monitoring student progress that 
worked particularly well? Increased efficiency? 

• Were there any interventions or supports that produced greater success than others? For which 
students? Should these interventions or supports be used more widely in the upcoming year? 

• Are there any new interventions or supports that the team feels should be offered in the future? 

• Are there any district or schoolwide changes that could or should be made to minimize the need 
for individualized supports? 

• How effective was the team in using monitoring to help identify and plan for student needs 
during the summer? 

• What else could the team do to improve this step? 
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Tool 22 - Step 6: End of the Year Reflection and Sustainability Planning Exercise 


Directions: Within your school or district team, reflect upon and answer the following questions. 


1. What are the most important changes you have implemented this past year as a result of your 
early warning implementation process and use of early warning data? Have you made any 
changes in how you: (1) review student data, (2) identify or support students, (3) assign 
interventions? What other changes have you made this year? 


2. What changes did you hope to implement as part of your early warning implementation, but 
were unable to this year? What impeded this implementation? 


3. List three to four implementation strategies that you used this year and that you are planning to 
sustain, refine, or scale up. 
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Tool 23 - Step 6: Early Warning Implementation Sustainability Action Plan 


Directions: The table provides an opportunity for your team to organize and plan for early warning implementation next year. List your three 
most important implementation strategies in column 1. Respond to the questions in columns 2 and 3 for each of the listed strategies. 


Column 1 

Column 2 

Column 3 

What is the step or change we want to sustain, 
refine, or scale next year? (See section 1, question 
3.) 

What have we done, or will we do, to best sustain this 
change for next year? What do we need to make this 
happen ( e.g ., funding, policy changes, staffing, culture 
shifts?) Who will be responsible for these steps? 

What potential obstacles or competing 
commitments might get in the way of this plan? 

How will we overcome these obstacles? 
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Tool 24 - Step 6: Early Warning Implementation Action Planning Tool 


Directions: The following template is designed to support your early warning implementation efforts for Step 6. Teams can identify the key 
objective or task, and then identify the resources available, actions needed responsible parties, and when the task needs to be completed. 

Key Tasks: 

• Plan time and approach for reflecting on student progress/outcomes and the implementation process. 

• Gather feedback from team members (e.g., distribute reflection questions). 

• Review team feedback and identify priority areas for discussion. 

• Identify areas that are working and explore areas that are not working and how the process could be improved. 

• Document any changes and articulate a plan for implementation. 

• Confirm new and continuing team members for the upcoming year and make plans for orienting new members. 


Objective or Task 

What do we have in place? 
What expertise and/or 
resources do we already 
have? 

What additional actions 

and resources are needed? 

Who will be responsible? 

What is the time frame? 
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Section 3. Additional Resources 

For additional information about early warning systems, dropout prevention, and data-driven improvement, please see the following resources. 

Massachusetts Resources 

■ Tools: 

• Massachusetts District Data Team Toolkit ( http://www.doe.mass.edu/apa/ucd/ddtt/Knowledge.pdf ): This toolkit is designed to help a 
district establish, grow, and maintain a culture of inquiry and data use that can inform decisions that impact teaching and learning, and 
ultimately improve the achievement of all students. 

• Learning Walkthrough ( http://www.doe.mass.edu/apa/dart/walk/ ): Learning walkthroughs are a systematic and coordinated method of 
gathering data to inform district- and school-level decision making. They involve establishing a Focus of Inquiry, and then engaging 
strategically selected teams of individuals in collaborative observations of classrooms with an emphasis on the interactions among 
teachers, students, and academic content (the instructional core). Learning walkthroughs can be a powerful means of helping educators 
learn more about the ways in which instructional practices support student learning and achievement. Evidence from learning 
walkthroughs can inform analyses of other data, such as EWIS data. 

• District Standards and Indicators Self-Assessment ( http://www.doe.mass.edu/apa/general/ ): The District Standards and Indicators self- 
assessment tool is meant to be used by district teams to support an in-depth collaborative process to measure, track, and enhance the 
effectiveness of the current district systems in a way that is focused on improving student achievement and consistent with the state's 
expectations for accountability and assistance. 

• Conditions for School Effectiveness Self-Assessment ( http://www.doe.mass.edu/apa/general/) : The Conditions for School Effectiveness 
self-assessment is designed to be used at the school level by leaders, those responsible for day-to-day instruction, and key stakeholders. 
It is a tool for conducting a scan of current practice, identifying areas of strength, and highlighting areas requiring greater focus. 

• The Behavioral Health and Public Schools Self-Assessment Tool ( http://bhps321.org ): This tool is designed for schools to assess current 
activities and strategies that the staff and programs engage in to create a supportive school environment. This tool is intended to assist 
with documenting current practices that support students' behavioral health at all intervention levels, ranging from the whole school 
community to individual students who require more intensive supports. It also examines the role of various school professionals and 
staff in providing these supports. 
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• Massachusetts Guide for Implementing Individual Learning Plans (ILPs) ( http://www.doe.mass.edu/ccr/) : This guide was designed for 
Massachusetts schools and districts to support the use of Individual Learning Plans (ILPs) for all students starting in middle school. The 
ILP is a student-directed, multi-year, dynamic tool that maps academic plans, personal/social growth, and career development activities 
while taking into account the student's unique, self-defined interests, needs, and goals for the attainment of postsecondary success. The 
ILP implementation may be a helpful to support students that are off-track for meeting an upcoming academic milestone. 

■ Information: 

• The Edwin website ( http://www.doe.mass.edu/edwin/) includes information about all of Edwin Analytics and Edwin Teaching and 
Learning. EWIS information includes the technical documents about risk model development and recorded EWIS tutorials. 

• The dropout reduction website ( http://www.doe.mass.edu/dropout/ ) includes resources and information related to a variety of dropout 
prevention, intervention, and recovery topics and promising practices. 

• The Massachusetts Tiered System of Support ( http://www.doe.mass.edu/mtss/ ) is a blueprint for school improvement that focuses on 
system structures and supports across a district, a school, and classrooms to meet the academic and nonacademic needs of all students. 
It was developed to help guide the establishment of a system that provides high-quality core educational experiences in a safe and 
supportive learning environment for all students and targeted interventions and supports for students who experience academic and/or 
behavioral difficulties and students who have already demonstrated mastery of the concept and skills being taught. 

• The Behavioral Health and Public Schools Framework (http://bhps321.org/bhps framework.doc) provides a structure to assist schools in 
creating safe, supportive environments and to increase support for students— particularly students with behavioral health challenges— 
through the provision of collaborative services. 

Other Resources 

■ Tools: 

• Early Warning System (EWS) High School Tool ( http://www.earlvwarningsystems.org/services/resources-tools) : Developed by AIR for the 
National High School Center, this tool is freely downloadable and Excel based. It is designed to allow users to identify and monitor students 
who show early warning signs that they are at risk for dropping out of high school. It assists users in monitoring the progress of students 
during the course of the school year (as early as the first 20 or 30 days of school and after every grading period). It relies on readily available 
student-level data (e.g., attendance, course failures, grade point average, and credit accumulation) that may be imported or entered into 
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the tool. As these data are entered, the tool automatically calculates indicators that are predictive of whether students will graduate or 
drop out. 


• Early Warning System (EWS) Middle Grades Tool ( http://www.earlvwarningsystems.org/services/resources-tools) : Developed by AIR for 
the National High School Center, this tool is freely downloadable and Excel based. It is designed to allow users to identify and monitor 
students who show early warning signs that they are at risk for dropping out of high school. This tool assists users in monitoring the 
progress of students during the course of the school year (as early as the first 20 or 30 days of school and after every grading period). It 
relies on readily available student-level data (attendance and course performance) that may be imported or entered into the tool. As 
these data are entered, the tool automatically calculates indicators that are predictive of whether students are on track to graduate 
from high school. 

■ Research and information: 

• Developing Early Warning Systems to Identify Potential High School Dropouts [http://www.earlywarningsystems.org/wp- 
content/uploads/documents/lssueBrief EarlyWarningSystemsGuide.pdf): This resource discusses the factors that help predict the 
probability that individual students will eventually drop out of high school before graduating and includes step-by-step instructions for 
building an early warning system. 

• Approaches to Dropout Prevention: Heeding Early Warning Signs With Appropriate Interventions 
( http://www.earlvwarningsystems.org/wp-content/uploads/documents/NHSC ApproachestoDropoutPrevention.pdf ): This report 
outlines steps that schools can take to identify students in need and provide the necessary support systems and relevant interventions 
to assist students in obtaining a high school diploma. Furthermore, the report discusses the use of early warning data systems to target 
interventions for groups and individual students, offers a variety of best practice approaches undertaken by higher performing high 
schools, and presents effective programs that are currently being implemented to stem the dropout problem. 
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Section 4. Appendices 

Appendix A. The Development of the Massachusetts EWIS Model 
Introduction 

The Massachusetts Early Warning Indicator System (EWIS) is based on several years of research 
examining state data and identifying those data that contribute to indicators that can most accurately 
predict the likelihood that a student will either achieve or miss key academic milestones. By helping to 
early identify students who are at risk, the EWIS allows educators to intervene and support struggling 
students to help them get back on track. 

Background 

The Massachusetts Department of Elementary and Secondary Education (ESE) created EWIS in response 
to district interest in the Early Warning Indicator Index that preceded EWIS; the index identified rising 
grade 9 students who were at risk of not graduating from high school. Per district request, EWIS was 
created to include the identification of students in grades 1-12 who are at risk of missing key academic 
outcomes or milestones. 

Academic Milestones and Age Groups 

ESE collaborated with American Institutes for Research (AIR) to develop the EWIS risk models for each 
grade level and grade cluster (also called age groups; e.g., early elementary school, late elementary 
school, middle school, and high school). Although there are some common indicators across age groups 
and grade levels, the risk models do vary by grade level. A team from ESE worked closely with AIR to 
determine the recommended models for each grade level, and an EWIS advisory group from ESE and 
other state agencies reviewed the research findings and discussed the key decisions. 

For each grade level, academic milestones have been identified that are developmentally appropriate, 
based on available state data, important to the success of Massachusetts students, and meaningful and 
actionable for adult educators who work with the students in each grade grouping. Exhibit A-l 
overviews the age groups and the academic outcomes or milestones. 
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Exhibit A-l. Massachusetts EWIS Age Groups, Grade Levels, and Academic Milestones 


Age Groups 

Grade 

Levels 

Academic Milestones 

(Expected Student Outcomes for Each Age Group) 

Early 

elementary 

1-3 

Reading by the end of grade 3: Proficient or Above Proficient on the 
grade 3 ELA State Assessment 

Late 

elementary 

4-6 

Middle school ready: Proficient or Advanced on the grade 6 ELA and 
mathematics State Assessment 

Middle grades 

7-9 

High school ready: Passing grades on all grade 9 courses 

High school 

10-12 

High school graduation: Graduating from high school in four years 


EWIS Risk Levels 

There are three EWIS risk levels: low, moderate, and high. These risk levels relate to a student's predicted 
likelihood for reaching his or her age group's defined academic milestone. A risk level quickly provides 
information to educators about the likelihood that a student will or will not meet an academic milestone. In 
other words, the risk level indicates whether a student is currently on track or off track to reach the 
upcoming academic milestone. For example, a student who is identified as being at low risk is predicted as 
likely to meet the academic milestone. 

A risk level is assigned to every student who was enrolled in a Massachusetts public school and for 
whom there are state level data in the prior year. The risk levels are determined using data from the 
previous school year. The risk levels are determined on an individual student basis, not on a student's 
relative likelihood of achieving an academic milestone when compared with other students. As a result, 
there are no set amounts of students for each risk level. For example, it is possible to have all students in 
the low-risk category. Exhibit A-2 overviews the risk levels. 


Exhibit A-2. Massachusetts EWIS Student Risk Levels 


Low Risk 

Likelv to reach the upcoming academic milestone. Approximately 90 percent of 

students who are at low risk will meet this academic milestone within each aae 

group. 

Moderate Risk 

Moderately at risk for not reaching the upcoming academic milestone. 

Approximately 60 percent of students at moderate risk meet this academic 

milestone within each age group. 

High Risk 

At risk for not reaching the upcoming academic milestone. Approximately 

25 percent of students at hiah risk meet this academic milestone within each aae 

group. 


To develop the EWIS risk levels, researchers used a rigorous statistical method 19 to create a valid EWIS 
model for each grade level from elementary through high school. The model allows users to identify 
students who are at risk of missing key academic milestones within the K-12 educational trajectory. 


Early Warning Implementation Guide 


Page 110 




Early warning systems cannot predict with 100 percent accuracy whether students will achieve 
academic outcomes. 


ESE intentionally designed EWIS to rely solely on data available from existing statewide collections. The 
data included come from several data sources: the Student Information Management System (SIMS) and 
Student Course Schedule (SCS) data collections, as well as assessment data results from State 
Assessments and the Assessing Comprehension and Communication in English State-to-State for English 
Language Learners (ACCESS for ELLs). Data included in the EWIS risk models are from the prior school 
year, to provide a risk level at the beginning of the next school year. EWIS is validated and updated 
annually as more recent years of data becomes available and to account for changes and shifts in data 
sources. Exhibit A-4 shows the indicators used for EWIS by grade levels. 


Exhibit A-4. Overview of the Final EWIS Model, by Grade Level 



Grade Level 

Age Group 

Early Elementary 

Late Elementary 

Middle School 

High School 

Academic Milestone 

Proficient or Advanced 
Proficient on Grade 3 
ELA State Assessment 

Proficient or Advanced 
on Grade 6 ELA and 
Mathematics State 

Assessment 

Pass All Grade 9 

Courses 

Graduate From High 
School in 

4 Years 

Indicators Included 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

Attendance rate 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

School move (in a single year) 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Number of in-school and out-of- 
school suspensions 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

ACCESS for ELLs level 




X 

X 

X 

X 

X 

X 

X 

X 

X 

ELA State Assessment 




X 

X 

X 

X 

X 

X 




Mathematics State Assessment 




X 

X 

X 

X 

X 

X 

X* 



Retained 


X 

X 

X 

X 

X 

X 

X 

X 

X 



Low income 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Special education level of need 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

ELL status 

X 

X 

X 










Gender 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Urban residence 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 



Overage for grade 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Schoolwide Title 1 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Targeted Title 1 

X 

X 

X 

X 

X 

X 







Mathematics course performance 







X 

X 

X 

X 

X 

X 

ELA course performance 







X 

X 

X 

X 

X 

X 

Science course performance 







X 

X 

X 

X 

X 

X 
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Section 4, Appendixes 



Grade Level 

Age Group 

Early Elementary 

Late Elementary 

Middle School 

High School 

Academic Milestone 

Proficient or Advanced 
Proficient on Grade 3 
ELA State Assessment 

Proficient or Advanced 
on Grade 6 ELA and 

Mathematics State 

Assessment 

Pass All Grade 9 

Courses 

Graduate From High 
School in 

4 Years 

Indicators Included 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

Social studies course performance 







X 

X 

X 

X 

X 

X 

Noncore course performance 







X 

X 

X 

X 

X 

X 


*The grade 10 EWIS risk model uses data from the grade 8 state assessment. 


Limitations and Uses for EWIS 

EWIS was created to provide educators with a tool to help predict which Massachusetts students may 
be most in need of additional interventions or supports because of the likelihood that they are currently 
off track for reaching an upcoming academic milestone. It is not an accountability metric. For each grade 
level, the probability of a student not meeting an academic outcome has been calculated based on 
existing, historical data. The predictive nature of these data and indicators is not absolute, and the EWIS 
risk level should never be the sole piece of data used for student planning. For example, major life 
experiences (a more stable or unstable environment for a child, parental [or guardian] marriage or 
divorce, birth and/or death of a family member, etc.) may have an impact that offsets the EWIS risk level 
assigned to a student. Also, school impacts, such as targeted student interventions, can move a student 
from being off track (high risk) to meeting an academic milestone. Research is clear that a variety of 
factors contribute to a student's likelihood of achieving key academic outcomes. 

The EWIS model is intended as a starting point for identifying and supporting students who may be at 
risk for not reaching an upcoming academic milestone. Because the EWIS data are calculated using data 
collected by ESE, districts and schools also may consider using local data to identify, diagnose, and 
support students. For more detailed information on EWIS development, refer to ESE's EWIS Technical 
Descriptions of Risk Model Development reports 
( http://www.doe.mass.edu/edwin/analytics/ewis.html) . 
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Appendix B. Accessing EWIS Data and Reports in Edwin 


Edwin Analytics is the comprehensive Massachusetts teaching and learning platform that is designed to 
provide state educators with integrated information and tools. Using Edwin Analytics, classroom 
teachers, principals, professional support personnel, and district leaders will be able to access a wide 
range of data reflecting attendance; course enrollment; students' growing acquisition of new 
knowledge, skills, and abilities; and college matriculation. EWIS is included in Edwin and can be accessed 
from the ESE security portal at https://gateway.edu.state.ma.us . 

Access and Supports 

EWIS offers Massachusetts public school districts information about student risk levels based on the 
prior year's data. This information is typically available to districts in late August for the start of each 
school year. 

EWIS Access. Districts can access information for any school or student in that district through Edwin 
Analytics. The security portal also can be found via the drop-down menu on the upper-right side of the 
ESE website ( http://www.doe.mass.edu/) . If you already have a user ID, the next step is to request 
access to Edwin Analytics from your district's directory administrator. Please see the Access and Security 
document on the Education Data Warehouse public website 

(http://www.doe.mass.edu/edwin/analytics/) for an explanation of these roles. For additional support 
on accessing Edwin and/or EWIS, please contact your district's directory administrator: 

http://www.doe.mass.edu/infoservices/data/diradmin/list.aspx 

EWIS Supports. To facilitate districts' use of EWIS, numerous supports are available, including the 
following: 

■ EWIS frequently asked questions ( http://www.doe.mass.edu/edwin/faq.html) 

■ Technical reports ( http://www.doe.mass.edu/edwin/analytics/ewis.html ) 

■ EWIS video tutorial ( http://www.doe. mass.edu/edwin/analytics/ewis.html?section=tutorials) 

Accessing and Maintaining Up-to-Date EWIS Data 

EWIS reports will be updated after the October SIMS is finalized each year. Prior to the October SIMS, 
students are listed in their anticipated grade level. After the October SIMS is finalized, the grade levels 
are updated to reflect any necessary changes (e.g., for students retained in a grade). It is important to 
note that districts can claim any new students to the district (or students attending a new school within 
the same district) at any point in time. By claiming new students frequently, districts can ensure that the 
EWIS reports adequately reflect all currently enrolled students. Districts are strongly encouraged to 
claim students during the summer so that the full list of current students is available to districts when 
the new EWIS data are available at the end of August. EWIS data also will be updated with corrected 
grade levels and school assignments when the October SIMS submission is finalized in late fall.. 
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Generating EWIS Reports in Edwin Analytics 


Within Edwin Analytics, districts and schools can choose to generate a range of EWIS reports or export 
EWIS student early warning data files to Excel and conduct analyses and generate reports on their own. 
Within Edwin, the EWIS data and reports provide districts and schools with three types of reporting 
options: 

■ District View (EW301) 

■ Graphical View (EW302) 

■ Risk Level Indicator Analysis (EW317) 

■ Subgroup Analysis (EW318) 

■ Student List (EW601) 

Within each option, districts and schools can filter the data in numerous ways (e.g., selecting specific 
schools, grade levels, and/or student subgroups) to gain varying perspectives on their data and help 
understand the scope, depth, and nature of student risk levels. 


EWIS Reports Located on the District & School Tabs 


EW301 Early Warning Indicator System (EWIS) District View 

EW301 provides aggregate EWIS student risk levels by school and grade levels. This report may also be 
run for the state as a whole, showing the breakdowns of student risk levels by grades. The student 
population in the report includes students who are currently claimed by the District/School and who 
were assigned a risk level for the selected school year. When viewing past school years, the students' 
school, grade, and risk level are based on the end of year information for the selected school year. With 
the appropriate security permissions, users can drill-down to the EW601 report to view the student- 
level information. 


EW302 Early Warning Indicator System (EWIS) Graphical View 

EW302 provides three graphical representations of the student risk levels of not achieving their next 
educational outcome as determined by the Early Warning Indicator System (EWIS) within a district. The 
first is a bar-chart of the risk levels for selected students currently claimed by the district/school and 
who were assigned a risk level for the selected school year. The selected students' risk levels are 
compared to the district and state risk levels. 

The second chart is a line graph (Section B) and depicts up to five years of student risk levels for the 
selected students. This graph shows the longitudinal risk levels for the selected student cohort. For 
example, if 5 th grade male students were selected, the graph shows the most recent student risk level 
breakdowns for these students, and also the student risk levels for these same students when they were 
in 4 th grade, 3 rd grade, etc. 
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The third chart is also a line graph (Section C) that depicts up to five years of student risk levels for the 
selected students. Unlike the graph in Section B, this graph shows the longitudinal risk of the selected 
student population profile. For example, if 5 th grade male students were selected, the graph shows the 
risk levels of 5 th grade male students this year, last year, two years ago, three years ago, and four years 
ago (presumably most of these students will be different each year). 


EW317 Early Warning Indicator System (EWIS) Risk Level Indicator Analysis 

EW317 is a tool to analyze the patterns of attendance, suspension, state assessment, and course 
performance data that are used to determine EWIS student risk levels. This report is run for a particular 
grade level. The report details, in a table as well as graphically, the indicator information for students in 
each risk level. Only indicators related to the selected grade level in the EWIS model are displayed. 


EW318 EWIS Subgroup Analysis 

EW318 compares, in aggregate, the EWIS student risk levels across student subgroups (e.g., 
race/ethnicity, high need subgroups. The report includes also a state comparison for the student 
subgroup populations. This data in this report is not suppressed and should be shared with caution. 


EWIS Report Located on the Students, Staff & Classroom Tab 


EW601 Early Warning Indicator System (EWIS) Student List 

EW601 provides a list of all students currently claimed in the district. The list includes the EWIS student 
risk level for each student, along with the indicators used to determine the risk level. Additional 
information about the student that was not used to determine their student risk level (e.g., their prior 
year's risk level, their race/ethnicity) is also provided. When viewing past school years, the students' 
school, grade, risk level, and risk indicators are based on the end of year information for the selected 
school year. Due to the student level information in this report the information should be shared with 
caution. 


Box B-l. Student Characteristics: Definition of Terms 


ELL indicates students who do not speak English or whose native language is not English and who are not 
currently able to perform ordinary classroom work in English. 

FELL indicates students identified as not currently ELLs but were ELLs within the past two years. 

Low income indicates students eligible for free or reduced-price lunch. 

SWD indicates students with disabilities (i.e., an individualized education program). 

High need indicates students identified as ELL, FELL, low income, or SWD. 
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Exhibit B-l. Sample Prompt Page to Select Filtering Options 


- District/ Schools 

District 

* | Malden (01650000) 



* 


Schools 

I? Beebe 
Ferryway 
Forestdale 
r~ Linden 

f Malden Early Learning Center 
f Malden High 
Salemwood 


Select all Deselect all 


Grade Level 


r ~ 01 

- 

r 02 


ro3 


r 04 


W 05 


r 06 


ro7 


I - 08 



Select all Deselect all 


Student Groups- 
Race/Ethnicity 


I - African American/Black 
I - Asian 

Hispanic or Latino 

\~ Multi-race, non-Hispanic or Latino 
American Indian or Alaskan Native 
I - Native Hawaiian or Pacific Islander 
White 

Select all Deselect all 


Gender 
High Needs 
Low Income 
ELL 
FELL 


~E\ 

~3 

~3 


Special Education Placement (Ages 6-21) 


\~ All Students with Disabilities 
Not a Special Education Student 
I - Exited Special Education in Current School Year 
Full Inclusion (>79%) 

|~ Partial Inclusion (40-79%) 

Substantially Separate (<40%) 

\~ Public Separate Day School 
Private Separate Day School 


jlI 

Select all Deselect all 


Submit 


Cancel 
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Exhibit B-2. Sample District View Report (EW301) 


£4 Keep this version » ^ PS ' S'' ^ Add ths report » 


edwin 


EW301 Early Warning Indicator System (EWIS) District View Sample District 

EWIS as of Jan-2014 for the 2013 2014 School Year for students currently enrolled in the 2013 2014 School Year 


School(s): All Schools within District 


Total Enrollment 

High 

Moderate 

Low 

N/A 

# 

# 

% 

# 

% 

# 

% 

# 

% 

Total for Report 

2,046 

301 

14.7 

332 

16.2 

1,412 

69.0 

1 

0.0 

School: x 

267 

75 

28.1 

64 

24.0 

128 

47.9 

0 

0.0 

Grade 01 

62 

4 

65 

25 

403 

33 

532 

0 

00 

Grade 02 

42 

8 

190 

14 

33.3 

20 

476 

o| 

00 

Grade 03 

46 

11 

239 

13 

283 

22 

478 

0 

00 

Grade 04 

58 

20 

34 5 

7 

12.1 

31 

534 

0 

00 

Grade 05 

59 

32 

542 

5 

85 

22 

373 

0 

00 

School: v 

135 

24 

17.8 

33 

24.4 

78 

57.8 

0 

0.0 

Grade 01 

20 

0 

00 

12 

600 

8 

400 

o| 

00 

Grade 02 

33 

4 

12.1 

7 

21 2 

22 

667 

0 

00 

Grade 03 

31 

4 

129 

6 

194 

21 

67 7 

0 

00 

Grade 04 

22 

5 

227 

3] 

136 

14 

636 

OJ 

00 

Grade 05 

29 

11 

379 

5 

172 

13 

44 8 

0 

00 

School: z 

216 

48 

22.2 

54 

25.0 

114 

52.8 

0 

0.0 

Grade 01 

44 

11 

250 

13 

295 

20 

455 

0 

00 

Grade 02 

47 

11 

234 

15 1 

31 9 

21 

44 7 

ol 

00 

Grade 03 

33 

4 

12.1 

4] 

12.1 

25 

758 

0 

00 

Grade 04 

45 

9 

200 

11 

244 

25 

556 

0 

00 

Grade 05 

47 

13 

277 

11 

234 

23 

489 

0 

00 
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Exhibit B-3. Sample Graphical View Report (EW302) 

Sample District 
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ed win 


EW302 Early Warning Indicator System (EWIS) Graphical View 

EW1S •• of Jon 2014 for iludtntt currently molM In Iho 201) 2014 School Yoor 
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Exhibit B-4. Sample Risk Level Indicator Analysis Report (EW317) 


ed 


School* 

|r 

p 

r 


X 

Y 

Z 


EWIS Risk Level Indicator Analysis 


PS’ S’ jMJtra.attr 

Sample District 

School Year MIS-14 


School: x 

Grade level „ 


Grad* 


Submit 


Report Overview 



Section A Attendance and Su*pen*4on 
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Section B: MCAS - ELA & Math 


B1.MCAS ELA 



■ 

H*h 

Moderate 

low 

• Solccted Students 

49 

43 

31 

% Advanced 

0 

2 

16 

% Pro6c.«nt 

it 

51 

W 

% Needs Improvement 

SI 

44 

0 

% Warning 

24 

2 

0 

% Missing MCAS 

6 

0 

0 


B2. MCAS Math 



High 

Moderate 

Low 

• Selected Students 

49 

43 

31 

% Advanced 

0 

0 

71 

% Protkient 

0 

44 

» 

% Needs Improvement 

45 

47 

0 

% Warning 

51 

9 

0 

% Missing MCAS 

4 

0 

0 


too* 

sou 

sou 

40U 

20U 

ou 



H % Advanced 
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H % Warning 
□ % Missing MCAS 
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Section C: Course Performance 


Cl. Passing Course by Subject 




100% 1 


thgh 

Modwat* 

loo 


• Sol*ct*d Students 

49 

43 

31 

80% - 





•0% - 

S Pass ELA 

76 

100 

100 

40% - 

% Pa** Math 

61 

too 

100 

% Pass Soane* 

61 

68 

100 

20% - 

% Pm* Social Stud** 

88 

100 

100 

0% - 

% Pass Non-Cor# 

63 

95 

100 




C2. Course Performance Overview 



H8fl» 

Motor at* 

Low 

- 

• Selected Student* 

49 

43 

31 

80% - 





Aej Court* Suited* Fated (*) 

14 

02 

00 

80% - 






% Faded No Court* Subfectt 

31 

84 

100 

40% - 

% Fated 1 Coot* Suti|*d 

31 

16 

0 


% Fated 2 Coo** Suited* 

18 

0 

0 

20% - 

% Filed 3 Coot* Suited* 

12 

0 

0 


% Fated 4 Coot* Subiect* 

4 

0 

0 

0% -- 
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Exhibit B-5. Sample Subgroup Analysis Report (EW318) 


rf l EWIS Subgroup Analysis 

Contains pfsonslly Ktont/ftsbh infofmsoon Wwh do not OistnMa 


Sample District 

School Ymt 2013-14 
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Exhibit B-6. Sample Student List Report (EW601) 

Note: due to the length of this report, the table sample is represented in two pieces below. 


R5- SI * [S&aubsutpac 


ed 


EW601 Early Warning Indicator System (EWIS) Student List 

EW1S as of Aug 2014 for the 2014-2015 School Year for students currently enrolled in the 2014-2015 School Year 


School(s): All Schools wtfhm District 

Grade Level: 08 


Middle School: Passing aO grade 9 courses 


EWtS Information 


ff 1 ’ 00 ' Grad* Ult Nam* F»« Nanw SASO 

Name 

Risk 
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Attend 
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Suspension 

Retained 

School 
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Income 

SPED 

Level 

Gender 

Urban 

Residence 
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School wide 

ACCESS 
for ELLs 

Sample Student 

ttgh 

89% 

0 

No 

No 

Free 

None 

Male 

Yes 

No 
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Exporting EWIS Data to Excel 


Some districts may want to expand their capacity to examine EWIS data beyond the reporting options offered through Edwin Analytics and 
therefore may choose to export their EWIS data into Excel as a separate EWIS data file. Districts can export EWIS data within Edwin Analytics to 
Excel by using the drop-down box on the upper-right side of the EWIS report screen. After the EWIS data are exported, districts and schools can 
combine these data with other sources of district and/or school data and/or generate a wider number of reports. 


Early Warning Implementation Guide 


Page 126 



Appendix C. Sample Early Warning Implementation Timeline 


Month 

Sample Activities 

August/Early 

September 
(Before start of 

school) 

Step 1: Get Organized 

O Determine team structure, composition, roles, responsibilities, and goals 

O Confirm access to EWIS data in Edwin Analytics 

O Identify student data sources to be used for monitoring and establish thresholds for flagging 

O Set data import and team meeting schedule for the year 

O Identify agenda items for the first team meeting of the school year 

September-October 

Step 2: Review EWIS Data 

O Convene first team meeting of the school year 

O Determine key questions and focus for data exploration 

O Generate EWIS reports to answer questions 

O Identify patterns in EWIS data and summarize key findings 

O Identify individuals or groups at risk, and/or grade/schoolwide areas of concern 

Step 3: Explore Underlying Causes 

O Consider possible factors contributing to student risk for identified students 

O Determine data collection plan (how/when/who will collect this additional data to confirm likely underlying causes) 

O Review additional data to discern likely underlying causes of risk 

O Confirm problem(s) to be addressed and potential interventions needed 

October-November 

Step 4: Take Action 

O Create inventory of available interventions and supports 

O Review student needs and available interventions and supports to identify opportunities and gaps. Check initial attendance 
data to ensure all at-risk students have been identified. 

O Confirm priorities for meeting student needs 

O Develop a plan for effectively using available resources to address individual, group, and schoolwide needs 

O Assign, document, and communicate about planned interventions and supports 
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Month 

Sample Activities 

December 

Step 5: Monitor Students and Interventions 

O Import monitoring data (e.g., course grades, discipline records, attendance) for first grading period 

O Review monitoring data to identify flagged students 

O Revisit Steps 3 and 4 (Explore Underlying Causes and Take Action) for students identified as at risk 

O Communicate with teachers, specialists, students, and families to assign and/or adjust interventions and supports 

January-March 

Steps 3- 5: Explore Underlying Causes, Take Action and Monitor Students and Interventions 

O Review monitoring data to identify students with risk flags 

O Revisit Steps 3 and 4 (Explore Underlying Causes and Take Action) for students identified as at risk 

O Communicate with teachers, specialists, students, and families to assign and/or adjust interventions and supports 

O Monitor progress of interventions and explore broader schoolwide changes to address the needs of many students 

O Import monitoring data (e.g., course grades, discipline records, and attendance) for third grading period 

April-June 

Steps 3- 5: Explore Underlying Causes, Take Action and Monitor Students and Interventions 

O Review monitoring data to identify students with risk flags and to guide planning for summer interventions and supports 

O Revisit Steps 3 and 4 (Explore Underlying Causes and Take Action) for students identified as at risk 

O Monitor progress of interventions and explore broader schoolwide changes to address the needs of many students 

O Communicate with teachers, specialists, students, and families to assign and/or adjust interventions and supports 

Summer 

O Step 6: Refine EWIS Process and Step 1: Get Organized 

O Reflect on team functioning and refine procedures for: importing data, reviewing data, and assigning interventions 

O Establish team composition, schedule and goals for upcoming year 

O Consider broader schoolwide changes to address the needs of many students 
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Appendix D. Targeted and Schoolwide Intervention Resources 

Your review of early warning data may suggest the need for targeted supports for students at risk, as well as broader schoolwide supports. 
Adopting targeted interventions can provide support tailored to the specific needs of groups of students at risk. Schoolwide interventions can 
help reduce risk for all students and, therefore, can be a cost-effective and proactive approach to addressing elevated levels of risk among all 
students within your school. The following table lists a number of resources that provide inventories of available interventions designed to 
address a range of student needs, from academic achievement to mental health. It is important to note that this is not a list of recommended 
interventions, nor is it an exhaustive list of available intervention resources. Rather, this list of resources can help you identify suitable 
interventions aligned with your school's and students' needs, and assist you in evaluating the interventions that are likely to meet the needs of 
students within your school's context. 

The targeted and schoolwide intervention resources are listed alphabetically in the intervention matrix. Within the matrix, you will find a brief 
description of each resource as well as some information regarding intervention types, the appropriateness of listed interventions by school 
levels, and the level of research review and rigor of evidence the resource/site required for including interventions. For information on how we 
classified resources' intervention type, level, and rigor of review, please see Appendix A— Matrix Key. 


Resource 

Brief Description 

Intervention Type 

Level 

Rigor of 
Review 

Best Evidence 

Encyclopedia 

■ Searchable database of educational programs, grouped by grade level 
and category (including a comprehensive school reform section); 
identifies top-rated programs. 

■ Users can access reports, educators' summaries, and educator guides 
discussing how programs were assessed for evidence of success. 

■ htto://www. bestevidence.org/ 

0Academic 
□Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 

□Middle 

□High 

M 

Blueprints for 

Healthv Youth 

Development 

■ Searchable database of 47 model and promising youth promotion 
programs, all of which have been reviewed and recommended by an 
independent panel. 

■ Users can filter interventions by target outcomes, target populations of 
students, program categories, program setting (including school), and 
risk and protective factors. 

■ htto://www. blueorintsDrograms.com/orogramSelector. oho 

□Academic 
0Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 

□Middle 

□High 

M 
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Resource 

Brief Description 

Intervention Type 

Level 

Rigor of 
Review 

Child Trends' What 

Works 

■ Searchable database of more than 600 out-of-school interventions 
targeting healthy child development, all of which have been reviewed 
by researchers. 

■ Users can filter their searches by age / grade level, by program setting 
(including a "school-based" option), by type of program, and by 
outcome. 

■ htto://www. childtrends.org/what-works/ 

0Academic 
□Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 

□Middle 

□High 

M 

Collaborative for 

Academic, Social, 

and Emotional 

Learning 

■ List of social-emotional learning programs that have been vetted and 
coded by researchers, with a particular focus on elementary-aged 
students. 

■ Users can access a profile of each intervention, which includes a 
summary of the intervention, the grade range covered, the program 
context (e.g., schoolwide) and evidence of the program's effectiveness. 

■ The site in is the process of developing a list of social-emotional 
learning programs targeting middle and high school students, with an 
anticipated release date of spring 2014. 

■ htto://www. casel.org/guide/orograms 

□Academic 
□Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 

0Middle 

□High 

L 

College & Career 

Readiness & 

Success Center at 

American Institutes 

for Research 

■ Searchable database that includes research articles, tools, and 
products, all of which have been screened for the extent of their 
research base. 

■ Users can search the database using key words. 

■ http://www.ccrscenter.org/ 

0Academic 

□Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 

0Middle 

□High 

L 

Earlv Warning 

Systems in 

Education 

■ Searchable library of materials focused on early warning systems as 
effective, research-based interventions for students at risk. 

■ Users can access a list of approaches to dropout prevention and 
featured examples of effective programs, and also can access tools for 
monitoring early warning system indicators. 

* http://www.earlvwarningsystems.org/resources-tools/implementation- 

□Academic 

□Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 

0Middle 

□High 

Nl 

and-research 
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Resource 

Brief Description 

Intervention Type 

Level 

Rigor of 
Review 

FindYouthlnfo 

Program Directory 

■ Searchable directory of evidence-based programs designed to reduce 
problem behaviors in youth . 4 

■ Users can customize searches by keyword and by main risk and/or 
protective factor; results can be sorted by age range and by rating 
(determined by study quality and program impact). 

■ http://www.findvouthinfo.gov/program-directorv 

□Academic 
□Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 

□Middle 

□High 

M 

National Center on 

Intensive 

Intervention 

■ Interactive chart listing studies of intensive academic interventions is 
featured; also hosts a list of behavioral and academic progress 
monitoring tools and a library of webinars around intensive 
intervention topics. 

■ Users can use the chart to examine intervention studies and find 
information about target population risk levels, the quality of the study 
design, the fidelity of implementation, whether study measures were 
targeted and/or broad, the effect size found, the intensity of the 
interventions, and whether additional studies have been conducted on 
the specific intervention. 

■ http://www.intensiveintervention.org/chart/instructional-intervention- 

□Academic 
□Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 
□Middle 
□ High 

M 


National Dropout 
Prevention Center / 

Network 

■ Searchable database of strategies and programs deemed effective for 
dropout prevention, in two formats: a list and a model programs 
database. 

■ Users can refine searches by grade level, evidence level, program 
emphasis, target groups (including a schoolwide option), risk factors, 
and more. 

■ http://www.dropoutprevention.org/effective-strategies 

□Academic 
□Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 

□Middle 

□High 

M 


4 The FindYouthlnfo Directory searches two databases simultaneously: the Teen Pregnancy Prevention Program Directory and crimesolutions.gov, the latter of 
which overlaps with the Office of Juvenile Justice and Delinquency Prevention’s Model Programs Guide . 
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Resource 

Brief Description 

Intervention Type 

Level 

Rigor of 
Review 

National Registry of 

Evidence-based 

Programs and 

Practices 

■ Searchable registry of more than 310 evidence-based interventions 
that promote mental health and safe behavior. 

■ Users can customize searches by age group, setting (school), outcome 
categories, and more. 

■ htto://www. nreoo.samhsa.gov/AdvancedSearch.asox 

□Academic 
□Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 

□Middle 

□High 

L 

Office of Juvenile 

Justice and 
Delinquency 
Prevention's Model 

Programs Guide 

■ Database of research-based youth intervention programs that are 
classified as "effective," "promising," or "no effects" by a panel of 
researchers. 

■ Users can review and search interventions by topic, as well as by focus 
(e.g., dropout prevention, school climate), age range, and protective 
and risk factors. 

■ htto://www. oiidD.gov/mDg/Tooic/Details/107 

□Academic 
□Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 

□Middle 

□High 

M 

Social and 

Emotional Learning 

Research: Evidence- 

■ List of social and emotional learning programs that have shown 
evidence of effectiveness, as supported by at least three independent 
evaluations or publications. 

* Users are provided information on program methods and associated 
outcomes, as well as a summary of the evidence base and citations for 
relevant studies. 

■ htto://www. edutoDia.org/sel-research-evidence-based-orograms 

□Academic 
□Social-emotional / 
Behavioral health 

□Special populations 

□Elementary 

□Middle 

□High 

M 

Based Programs 
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Targeted and Schoolwide Interventions Matrix Key 


Table 1. Intervention Type provides a description for each of the types of interventions covered by our listed 
resources. 


Intervention Type 

Definition 

Academic 

Interventions targeting academic achievement, 21st century skills, study skills, and 
vocational / life skills 

Social-emotional / 
Behavioral health 

Interventions targeting social and emotional development, mental health, enhanced 
behavioral supports and / or the promotion of healthy behaviors and the reduction of risky 
behaviors (e.g., delinquency, drug use) 

Special populations 

Interventions targeting students with higher needs for support, such as students with 
disabilities or limited English proficiency 


Table 2. School-Level Classification provides the grade levels for which the listed resource provides interventions. 


Level 

Definition 

Elementary 

Targeted at students in kindergarten through Grade 5 

Middle 

Targeted at students in Grades 6 through 8 

High 

Targeted at students in Grades 9 through 12 


Table 3. Rigor of Review indicates the extent to which research on the intervention is reviewed and included in the 
resource. For further information on the criteria for inclusion established by a given source, follow the embedded 
link in the Rigor of Review column. 


Rigor of Review 

Definition 

No Information 
(Nl) 

There is limited or no information about how interventions are identified for inclusion in the 
list of interventions. In some instances, this may be user driven. 

Low (L) 

Research on each intervention is reviewed. Interventions may be included with research- 
based evidence of positive impact determined solely by the research authors (i.e., without an 
independent review). 

Moderate (M) 

Research on interventions is reviewed. The quality and rigor of the research are confirmed by 
an independent source before it is included in the assessment of intervention 
impact/outcomes. However, the standard for research to be included is varied in terms of the 
size of the study and the rigor of the methods. 

High (H) 

Research on interventions is reviewed and confirmed by an independent source. Only large- 
scale studies and randomized controlled trials are included as evidence of an intervention's 
impact. 
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